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ABSTRACT 


The  use  of  simulation  to  train  watchstanders  in  marksmanship  would  provide  a  valuable 
and  flexible  training  asset  to  the  Navy,  resulting  in  minimal  lost  training  opportunities 
due  to  operational  commitments  at  sea.  We  hypothesized  that  (1)  simulation-based 
marksmanship  training  would  transfer  to  live  fire  better  than  dry  fire  training,  and  (2)  the 
experimental  (simulation)  group  would  have  a  better  chance  of  retaining  their 
marksmanship  skills  than  the  control  group  after  two  or  four  weeks  with  no  instruction. 

Thirty-four  active  duty  military  volunteers  were  randomly  assigned  to  receive 
either  simulation  training  using  the  Indoor  Simulated  Marksmanship  Trainer  (ISMT)  or 
standard  navy  marksmanship  training  and  given  either  a  two-  or  four-week  gap  between 
training  and  final  live  fire  events.  Main  measures  of  marksmanship  performance  were 
mean  point  of  impact  (MPI)  of  group  shots  and  scores  on  the  standard  Navy  Handgun 
Qualification  Course. 

Results  partially  supported  the  hypotheses.  The  simulation  group  showed  greater 
improvement  in  MPI  than  the  control  group  from  baseline  to  live  fire.  However,  no 
significant  differences  were  found  between  the  two-  and  four-week  gaps  in  either  case 
tested,  suggesting  a  longer  time  gap  is  needed  to  test  skill  retention.  Results  suggest  that 
simulation  training  is  as  effective  as  standard  navy  marksmanship  training  and  would 
benefit  the  Navy  to  incorporate  ISMT  as  an  at-sea  marksmanship  trainer. 
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I.  INTRODUCTION 


A.  PROBLEM  STATEMENT 

The  Navy  has  access  to  41  Indoor  Simulated  Marksmanship  Trainers  (ISMTs). 
The  authors  informally  interviewed  approximately  50  Navy  officers  with  recent 
experience  in  Fleet  concentration  areas  where  the  Navy  ISMT  systems  are  located  and 
only  one  knew  of  the  existence  of  ISMT.  This  suggests  that  the  Navy  does  not 
adequately  communicate  the  availability  of  ISMT  to  sailors  in  the  Fleet.  As  a  result, 
many  commands,  both  at  sea  and  ashore,  do  not  take  advantage  of  training  watchstanders 
with  ISMT,  but  instead  rely  on  standard  Navy  marksmanship  training  (SNMT). 
According  to  the  Department  of  the  Navy’s  Small  Arms  Training  and  Qualification 
instruction,  OPNAVINST  3591.  IF  (Chief  of  Naval  Operations,  2009,  Enclosure  3,  1): 

All  personnel  must  requalify  with  live  fire  annually.  All  personnel  are 
also  required  to  undergo  semi-annual  sustainment  training  between 
qualification  shoots,  not  to  exceed  8  months  after  live  fire  qualification.  If 
available,  sustainment  training  shall  be  completed  on  a  simulator. 

Oftentimes  requalification  and  sustainment  training  are  based  on  operational 
requirements,  resulting  in  lost  training  opportunities.  Currently,  most  watchstanders  only 
handle  a  firearm  when  they  check  in  and  out  of  the  armory  which  is  typically  two  to  three 
times  a  month  on  duty  days.  The  training  currently  being  used  by  the  Navy  focuses  on 
how  to  properly  respond  to  a  force  protection  threat,  but  assumes  the  watchstander  knows 
how  to  properly  use  a  firearm  based  on  dry  fire  and  PowerPoint  training.  This 
assumption  leads  to  a  concern  that  personnel  may  know  the  required  actions,  but  may  not 
have  the  marksmanship  skills  required  to  effectively  employ  their  weapon. 

The  concept  is  simple:  until  a  person  has  received  enough  practice  discharging 
rounds  down  range  to  reach  the  required  level  of  accuracy  and  proficiency,  it  is  not 
certain  that  they  will  be  effective  with  firearms  during  an  actual  force  protection  situation 
(Getty,  2008).  A  force  protection  situation  is  any  instance  in  which  personnel  or 
organizations  intend  to  advance  their  political  agenda  against  the  United  States  (U.S.)  by 
harming  U.S.  assets  (Naval  Surface  Forces  Command,  2008).  Effective  use  of  a  firearm 
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is  an  extremely  important  factor  in  suppressing  a  threat  if  a  force  protection  situation  is 
detected.  Navy  personnel  are  trained  to  make  the  proper  announcements  to  alert  the  crew 
and  give  the  proper  commands  to  deal  with  an  attacker,  but  if  the  watchstander  cannot  hit 
the  target  when  required,  then  the  entire  antiterrorism  and  force  protection  training 
system  fails. 

Currently,  the  Navy  marksmanship  training  program  teaches  fundamental 
marksmanship  skills  via  dry  fire  and  PowerPoint  to  all  watchstanders  and  includes 
antiterrorism  and  force  protection  training  that  incorporates  real-world-type  scenarios,  but 
uses  mock  weapons.  A  small  number  of  shipboard  watchstanders  receive  specialized 
training  as  a  reaction  force,  to  include  Ship’s  Reactionary  Force  Advanced  training,  to 
combat  various  shipboard  threat  situations.  Visit  Board  Search  and  Seizure  teams  get 
more  advanced  real-world,  scenario-type  training  to  include  role  playing  and  paint  ball, 
but  accounts  for  only  a  small  number  of  service  members.  The  use  of  simulation  to  train 
watchstanders  is  minimally  used,  but  would  provide  a  valuable  and  flexible  training  asset 
to  the  Navy,  resulting  in  minimized  lost  training  opportunities  due  to  operational 
commitments.  It  is  reasonable  to  believe  that  personnel  would  be  more  likely  to  protect 
the  ship  during  an  attack  if  commands  regularly  utilized  available  simulators  or  placed 
simulators  onboard  ships  so  that  personnel  could  practice  their  marksmanship  skills  on  a 
regular  basis. 

B.  STUDY  PURPOSE 

This  thesis  explores  whether  simulation-based  marksmanship  training  with  the 
ISMT  transfers  to  live  fire,  and  also  attempts  to  provide  insight  into  how  long  any 
beneficial  effects  of  simulation  training  last.  This  research  also  investigates  whether  the 
use  of  simulation,  combined  with  live  fire,  can  provide  a  more  robust  method  than 
SNMT.  The  course  of  fire  used  in  this  thesis  was  the  standard  Navy  Handgun 
Qualification  Course  (NHQC)  firing  sequence  defined  in  OPNAVINST  3591.  IF  (Chief 
of  Naval  Operations,  2009).  By  providing  the  means  to  practice  a  perishable  skill  like 
marksmanship,  the  Navy  may  have  fewer  incidents  or  casualties  (Getty,  in  progress). 
The  research  questions  addressed  in  this  thesis  are: 
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•  Does  ISMT  result  in  skill  transfer  from  the  virtual  environment  to  actual 
proficiency  in  marksmanship  performance? 

•  Can  ISMT  be  used  as  an  effective  part-task  trainer  to  improve 
performance  with  the  9mm  Berretta  (M9)  on  the  NHQC?  Specifically: 
Will  participants  in  the  experimental  (simulation)  group  have  a  better 
chance  of  retaining  what  they  learned  after  two  weeks  or  four  weeks  of  no 
instruction,  than  participants  in  the  control  group  (SNMT)  when  doing 
live  fire? 

C.  LITERATURE  REVIEW 

With  the  increased  operational  tempo,  ships  are  going  out  to  sea  more  frequently. 
According  to  Navy  Times,  deployments  that  were  traditionally  six  months  in  duration 
have  now  increased  to  between  8  and  10  months  (Faram  et  al.,  2012).  Given  that 
deployments  now  entail  at  least  eight  months,  it  is  often  difficult  for  personnel  to 
maintain  current  sustainment  requirements.  More  time  at  sea  means  less  time  in  port  for 
personnel  to  train  in  basic  skills  like  marksmanship.  In  order  to  meet  the  scheduling 
demands  and  operational  commitments  at  sea,  oftentimes  operational  requirements 
prevent  any  type  of  topside  marksmanship  training  while  at  sea.  The  Navy  currently  does 
not  have  marksmanship  simulators  onboard  ships,  but  having  an  onboard  simulator 
within  the  skin  of  the  ship  could  allow  personnel  to  circumvent  the  necessity  to  go 
topside  to  conduct  marksmanship  training  and  provide  an  opportunity  to  train  basic 
marksmanship  skills  at  sea. 

Little  is  known  about  how  ISMT  training  transfers  to  live  fire.  LTC  Yates,  United 

States  Marine  Corps  (USMC),  studied  the  effectiveness  of  the  ISMT  to  train  Marines  in 

marksmanship  fundamentals  (2004).  Using  28  participants  at  Marine  Corps  Recruit 

Depot,  San  Diego  during  the  initial  rifle  qualification  using  the  USMC  course  of  fire,  a 

side-by-side  comparison  was  conducted  to  measure  the  performance  of  Marines  trained 

in  ISMT  compared  to  Marines  who  were  trained  using  dry  fire,  culminating  in  a  final  live 

fire  qualification.  The  results  showed  no  significant  difference  in  group  shots  and  scores; 

in  other  words,  the  Marines  who  trained  with  ISMT  performed  as  well  as  those  who 

trained  without  ISMT.  LTC  Yates  concluded  that  there  was  a  lack  of  evidence  to  support 
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ISMT  being  used  as  the  only  training  mechanism  without  also  providing  expert 
instruction.  The  importance  of  this  study  was  that  the  ISMT  training  was  as  effective  as 
dry  fire. 

The  U.S.  Army  Research  Laboratory  conducted  a  training  transfer  study,  A 
Comparison  of  Live  and  Simulated  Fire  Soldier  Shooting  Performance  ( ARL-TR-4234, 
2007),  using  the  Dismounted  Infantry  Survivability  and  Lethality  Testbed 
simulation-based  training  to  determine  the  transfer  to  live  training.  Twelve  participants 
completed  a  course  of  fire  using  the  M16  to  fire  at  18  pop-up  target  silhouettes.  Firing 
ranges  consisted  of  75-,  100-,  150-,  200-,  250-,  and  300-meter  targets  from  a  kneeling, 
foxhole-supported  position.  Each  participant  was  exposed  to  10  trials  of  simulation  and 
live  fire,  and  hit  percentage,  shot  reaction  time,  shooting  performance,  and  radial  aiming 
error  were  assessed  under  both  conditions.  Both  simulation  and  live  fire  showed  similar 
hit  percentages,  indicating  that  participants’  maintained  performance  from  simulation  to 
live  fire.  In  live  fire,  each  shot  took  less  time  and  reaction  times  were  faster  compared  to 
the  shots  and  reaction  times  in  the  simulation.  The  difference  in  reaction  times  may  be 
attributed  to  many  factors  including  human  attributes  such  as  rifle  movement  with  a  live 
weapon  or  outside  weather  conditions.  The  notion  that  there  is  a  strong  relationship 
between  basic  marksmanship  skills  in  simulation  and  live  fire  is  supported  by 
this  research. 

The  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences’  Research 
Report  1761  (2000)  compared  simulated  marksmanship  training  to  standard  Army 
instruction  with  a  final  live  fire  qualification  course  of  fire.  One  hundred  eighty-four 
participants  trained  with  simulation,  while  two  hundred  two  participants  trained  with 
standard  Army  instruction.  All  participants  completed  1 1  periods  of  instruction,  ranging 
from  introduction  to  basic  rifle  marksmanship  and  mechanical  training  to  practice  firing, 
leading  up  to  firing  for  qualification.  The  dependent  variables  were  the  number  of  targets 
hit  and  rounds  fired  during  training  and  qualification  fire.  Results  indicated  that  the  use 
of  simulation  for  basic  rifle  marksmanship  training  reduced  the  number  of  live  rounds 
fired,  increased  the  number  of  participants  firing  to  standard  requirements,  and  increased 
the  number  of  target  hits.  Simulation  did  not  improve  record  fire  qualification  scores. 
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The  results  of  the  Research  Report  1761  (Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences,  2000)  suggest  that  simulation-based  marksmanship  training  would 
improve  initial  entry  rifle  marksmanship  performance,  while  saving  ammunition. 

The  above  studies  have  addressed  Marine  or  Army  marksmanship  training;  there 
has  been  very  little  research  directed  at  Navy  marksmanship  training,  specifically  with  a 
time  gap  between  training  and  qualification  course  of  fire.  Getty  (2008)  compared 
simulation  marksmanship  training  using  the  ISMT  to  SNMT  with  a  one-week  gap 
between  training  and  qualification  course  of  fire.  A  between-groups  study  was  conducted 
with  random  selection  of  participants  blocked  by  previous  marksmanship  experience  in 
which  half  of  the  participants  received  training  in  the  ISMT,  while  the  other  half  received 
SNMT.  The  dependent  variables  were  diameter  size  of  average  group  shots,  diameter 
size  of  group  shots  and  length  of  mean  point  of  impact  to  center  zeroing  point  for  the 
3-,  7-,  and  15-yard  lines,  and  overall  scores  on  the  NHQC  course  of  fire.  Getty’s  findings 
show  there  was  no  change  in  participants’  marksmanship  performance  and  scores 
regardless  of  whether  they  received  the  ISMT  training  or  the  SNMT,  except  for  the 
7-yard  line.  For  the  7-yard  line,  participants  who  received  the  ISMT  training  showed 
greater  improvements  in  performance  than  participants  who  received  SNMT. 
Exploratory  analyses  suggest  that  simulation  training  may  be  most  beneficial  for  less 
experienced  shooters.  Additionally,  there  was  no  significant  difference  in  skill  retention 
after  one  week  between  the  ISMT  participants  and  SNMT  participants,  suggesting  that  a 
longer  period  of  time  is  needed  to  detect  group  differences  in  the  retention  of  skills. 

In  summary,  the  studies  by  LTC  Yates  (2004),  ARL-TR-4234  (U.S.  Army 
Research  Laboratory,  2007),  and  Research  Report  1761  (U.S.  Army  Research  Institute 
for  the  Behavioral  and  Social  Sciences,  2000)  show  that  simulation  is  at  least  as  effective 
in  marksmanship  training  as  current  standard  Army  and  Marine  marksmanship  training 
and  can  be  a  cost-effective  way  to  train.  What  is  not  known  is  the  effect  that  simulation- 
based  training  has  on  live  fire  compared  to  the  effects  of  SNMT.  The  main  difference 
between  Navy,  Marine,  and  Army  marksmanship  training  is  the  weapon  that  is  being 
fired,  the  distance  fired,  and  the  rate  of  fire.  In  addition,  Getty  (2008)  attempted  to 
compare  simulation  training  to  SNMT,  with  only  a  one-week  gap  between  training  and 
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qualification  course  of  fire.  It  is  still  not  known  how  long  any  beneficial  effects  last  from 
simulation-based  training  to  live  fire.  Thus,  the  purpose  of  this  thesis  was  to  attempt  to 
address  these  two  important  gaps  in  knowledge. 

D.  OBJECTIVES 

Currently,  the  Navy  has  access  to  41  ISMT  simulators,  of  which  none  are  onboard 
ship.  The  existence  of  the  ISMT  simulators  is  not  well  communicated  and  simulation  is 
not  used  to  its  fullest  capacity,  which  results  in  many  lost  training  opportunities.  With  no 
simulators  located  onboard  ship,  marksmanship  training  is  dependent  on  operational 
requirements  while  at  sea  and  is  often  neglected  due  to  other  shipboard  evolutions  such  as 
flight  or  wet  well  operations.  As  a  result,  shipboard  personnel  may  not  receive  required 
sustainment  training,  thus  weakening  the  ship’s  first  line  of  defense — watchstanders. 

The  objective  of  this  thesis  was  to  determine  if  the  use  of  simulation,  combined 
with  live  fire,  can  provide  a  more  robust  method  than  SNMT  for  effectively  training 
watchstanders.  ISMT  was  used  as  a  part  task  trainer  to  teach  basic  marksmanship  skills 
to  the  experimental  group,  while  dry  fire  was  used  to  teach  basic  marksmanship  skills  to 
the  control  group,  immediately  followed  by  the  NHQC  firing  sequence  in  ISMT  by  both 
groups.  With  either  a  two-week  or  four-week  gap  after  training,  all  participants 
completed  a  live  fire  event  consisting  of  the  NHQC  firing  sequence.  Main  measures  of 
marksmanship  performance  were  MPI  of  group  shots  and  scores  from  the  NHQC 
firing  sequence. 

E.  RESEARCH  QUESTIONS  AND  HYPOTHESES 

1.  Question  One 

Does  the  ISMT  accomplish  skill  transfer  from  the  virtual  environment  to  actual 
live  fire  proficiency  in  marksmanship  performance? 

HOi:  There  will  be  no  group  differences  in  the  participants’  MPI  of  shots  and 
scores  in  the  NHQC  firing  sequence  from  baseline  to  live  fire  event. 
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HAj:  Participants  who  receive  simulation-based  training  in  ISMT  will  have  a 
greater  improvement  in  MPI  of  shots  and  scores  in  the  NHQC  firing  sequence  when 
comparing  the  difference  between  baseline  and  live  fire  than  participants  who 
receive  SNMT. 

2.  Question  Two 

Can  the  ISMT  be  used  as  an  effective  part-task  trainer  to  improve  performance 
with  the  M9  on  the  NHQC?  Specifically,  will  participants  in  the  simulation  group  have  a 
better  chance  of  retaining  what  they  learned  after  two  weeks  or  four  weeks  of  no 
instruction,  than  participants  in  the  control  group,  when  performing  live  fire? 

HOi:  Participants  in  each  group  will  be  equally  likely  to  maintain  their  MPI  of 
shots  and  scores  on  the  NHQC  two  weeks  and  four  weeks  after  the  training  day. 

HAi:  Participants  who  receive  simulation-based  training  in  ISMT  will  be  more 
likely  than  those  in  the  control  group  to  maintain  their  MPI  of  shots  and  scores  in  the 
NHQC  two  weeks  and  four  weeks  after  the  training  day. 
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II.  METHOD 


A.  OVERVIEW  OF  RESEARCH  DESIGN 

This  thesis  closely  follows  the  procedures  used  by  Getty  (2008).  Getty’s 
experiment  explored  whether  simulation-based  marksmanship  training  with  the  ISMT 
improved  marksmanship  performance  to  a  greater  extent  than  SNMT.  A  between-groups 
study,  with  a  random  selection  of  volunteers  (blocked  by  previous  marksmanship 
experience),  was  conducted.  The  independent  variables  were  the  two  different  training 
programs.  The  control  group  received  SNMT  that  covered  fundamentals  of 
marksmanship,  followed  by  untimed  dry  fire,  a  practice  qualification  that  implemented 
time  limits.  The  simulation  group  received  marksmanship  training  enhanced  by 
simulation  of  the  feedback  that  ISMT  provides  (Getty,  2008). 

The  experimental  design  for  this  thesis  utilized  a  between-groups  study  of  active 
duty  military  volunteers  randomly  selected  to  complete  either  SNMT  for  the  control 
group  or  simulation  training  conducted  using  the  system  feedback  provided  from  ISMT 
for  the  simulation  group.  Marksmanship  performance  was  measured  by  MPI  and  score 
using  the  NHQC  firing  sequence.  This  thesis  research  was  approved  by  the  Naval 
Postgraduate  School  (NPS)  Institutional  Review  Board  (IRB);  IRB  approval  number 
NPS  .20 11.01 02-IR-CONV  -A. 

B.  PARTICIPANTS 

All  participants  were  active  duty  U.S.  military  members  and  all  were  either 
students  or  faculty  at  NPS.  A  total  of  34  participants,  17  in  the  simulation  group  and 
17  in  the  control  group,  took  part  in  this  study.  Of  the  34  initial  participants,  33 
completed  the  pre-training  demographics  survey.  Tables  1  and  2  characterize  the 
simulation  and  control  group  participants’  general  demographic  and  previous 
marksmanship  experience.  Of  note,  no  significant  differences  between  simulation  and 
control  groups  were  found  for  any  demographic  or  previous  marksmanship  experience 
items. 
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Table  1.  General  Statistics  from  Participants’  Demographic  Surveys 


Demographics  Survey  (General) 

Control 

Simulation 

Age  (%):  Years 

24-29 

19 

24 

30-36 

31 

35 

37-41 

31 

35 

42+ 

19 

6 

Gender  (%): 

Male 

94 

94 

Female 

6 

6 

Corrected  Vision  (%): 

Yes 

38 

12 

No 

62 

88 

Height  (in): 

Mean 

70.38 

69.47 

Standard  Deviation 

2.42 

2.00 

Weight  (lbs): 

Mean 

190.69 

186.29 

Standard  Deviation 

27.56 

17.43 

Years  of  Service: 

Mean 

13.66 

11.79 

Standard  Deviation 

7.92 

4.83 

Branch  of  Service  (%): 

Army 

0 

18 

Navy 

68 

76 

Air  Force 

13 

6 

Marines 

13 

0 

Coast  Guard 

6 

0 

Pay  Grade  (%): 

E-5 

6 

6 

0-2 

6 

0 

0-3 

50 

53 

0-4 

13 

17 

0-5 

13 

24 

0-6 

12 

0 

Job  Specialty  (%): 

Surface  Warfare 

13 

24 

Aviation 

25 

24 

Ground  Element 

0 

11 

Intelligence 

31 

29 

Engineering 

31 

12 
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The  general  information  gathered  from  the  demographic  surveys  showed  no 
significant  differences  between  the  control  and  simulation  groups  in  any  of  the 
characteristics  collected  (based  on  two-tailed  a  =  0.05,  all  p  >  0. 1 18). 


Table  2.  Firearm  Specific  Statistics  from  Participants’ 

Demographic  Surveys 


Demographics  Survey  (Firearms) 

Control 

Simulation 

Dominant  Shooting  Hand  (%): 

Ambidextrous 

6 

6 

Right 

88 

88 

Left 

6 

6 

Self-Reported  Proficiency  (%): 

Novice 

19 

23 

Marksman 

38 

29 

Sharpshooter 

13 

24 

Expert 

31 

24 

Last  Fired  a  Weapon  (Years): 

Mean 

3.20 

1.35 

Standard  Deviation 

7.73 

1.45 

Earliest 

Jun  80 

Jun  07 

Latest 

Nov  11 

Oct  11 

Formal  Marksmanship  Training  (%): 

Yes 

81 

76 

No 

19 

24 

Last  Formal  Training  (Years): 

Mean 

8.66 

6.06 

Standard  Deviation 

11.49 

5.39 

Earliest 

Jun  80 

Jul  95 

Latest 

May  11 

Oct  11 

Performance  on  Last  Qualification  (%): 

Never  Shot  for  Qualification 

6 

12 

Marksman 

6 

6 

Sharpshooter 

13 

23 

Expert 

75 

59 

Last  Qualification  (Years): 

Mean 

6.28 

4.78 

Standard  Deviation 

8.71 

4.34 

Earliest 

Jun  80 

Jan  97 

Latest 

May  11 

Mar  11 
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The  firearm-specific  information  gathered  from  the  demographic  surveys  shows 
no  significant  differences  between  the  control  and  simulation  groups  in  any  of  the 
characteristics  collected  (based  on  two-tailed  a  =  0.05,  all  p  >  0. 180). 

C.  EQUIPMENT 

1.  ISMT 

ISMT  is  currently  employed  by  the  USMC  as  a  portable,  stand-alone 
marksmanship  trainer.  ISMT  is  a  “three  dimensional  simulation  based  trainer  for  indoor 
use,  capable  of  instructing  in  basic  and  advanced  marksmanship,  shoot/no- shoot 
judgment,  combat  marksmanship,  and  weapons  employment  tactics”  (U.S.  Marine  Corps 
Concepts  in  Programs,  2008,  p.  214).  ISMT  has  the  capability  to  use  a  wide  variety  of 
weapons,  including  the  .50cal.  machinegun  (M2),  9mm  Beretta  (M9),  5.56mm  service 
rifle  (Ml 6),  7.62mm  machinegun  (M240),  and  many  more.  The  ISMT  located  at  NPS 
can  train  up  to  two  individuals  at  a  time.  ISMT  has  the  unique  capability  to  “provide 
immediate  feedback  to  the  instructor  and  trainee  on  weapon  trigger  pull,  cant  position, 
barrel  movement,  rifle  butt  pressure,  tracing  of  the  muzzle  on  a  weapon  prior  to  and  after 
shoot,  and  grouping”  (Getty,  in  progress,  p.  3).  Immediate  feedback  is  vital  in 
marksmanship  training  because  it  provides  the  opportunity  for  participants  to  adjust  the 
weapon  accordingly,  thereby  greatly  improving  accuracy.  Just  as  important,  ISMT 
records  muzzle  movement  and  displays  the  recording  as  part  of  the  trace  profile  feature. 
The  trace  profile  feature  records  0.2  seconds  before  the  trigger  squeeze,  the  actual  trigger 
squeeze,  and  0.2  seconds  post  trigger  squeeze.  The  data  displayed  from  the  trace  profile 
feature  allows  the  instructor  to  critique  the  participant’s  technique,  which  would  be 
nearly  impossible  to  do  via  the  naked  eye. 

The  ISMT  lab  is  located  in  a  30-foot  trailer  on  the  NPS  campus.  The  front  section 
of  the  trailer  contains  the  stored  weapons  and  two  Windows-based  computers  that  run  the 
ISMT.  The  weapons  used  in  this  research  were  tethered  to  the  CO2  cylinder  bank  and  the 
ISMT  operating  system,  with  input  leads  consisting  of  a  hose  for  CO2  and  a  wire  harness 
to  provide  audio  and  system  feedback.  The  main  section  of  the  trailer  contains  the 
instructor  control  panel  and  the  instructor’s  personal  laptop,  just  forward  of  the  firing 
line.  The  instructor’s  laptop  is  connected  to  two  small  speakers  mounted  just  behind  the 
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firing  line  in  order  to  play  the  recorded  training  instructions.  Also  behind  the  firing  line 
is  the  CO2  cylinder  bank  that  provides  the  recoil  for  the  weapons.  The  projector  that 
displays  the  video  for  the  scenario  and  the  camera  that  reads  the  laser  from  the  weapons 
are  mounted  on  the  ceiling  in  the  main  section  of  the  trailer.  The  large  projector  screen 
that  displays  the  scenarios  and  the  subwoofers  that  provide  audio  for  realistic  weapon 
firing  sound  effects  are  located  at  the  end  of  the  trailer. 

2.  Live  Fire  Equipment 

In  order  to  complete  the  live  fire  portion  of  this  thesis,  the  following  equipment 
was  used:  six  M9s,  six  belts  with  holsters  and  magazine  pouches,  36  magazines  (six  per 
weapon  to  facilitate  the  firing  sequence),  three  belts  with  holsters  and  magazine  pouches 
for  ISMT,  2,000  rounds  of  ammunition,  96  blue  U.S.  Treasury  Transitional  Targets 
(Transtar  II)  for  the  live  fire  event,  and  eye  and  hearing  protection  for  each  participant. 

Live  fire  was  completed  at  the  Laguna  Seca  Peace  Corps  Range.  The  range  is 
outdoors  and  offers  25  lanes.  Mandatory  personal  protective  equipment,  including  eye 
protection  (side  shielded/wraparound),  hearing  protection  (plugs/muffs),  and  protective 
clothing  (long  sleeves),  was  worn  by  all  participants  while  participating  in  the  live  fire 
event.  Two  hospitals,  Community  Hospital  of  the  Monterey  Peninsula  and  Salinas 
Valley  Memorial  Hospital,  were  in  close  proximity  to  the  Laguna  Seca  Pistol  Range  to 
address  any  medical  incident  that  may  have  arisen  during  this  study.  The  NPS  IRB 
assigned  a  medical  monitor  to  this  study  to  address  any  medical  situation.  The  medical 
monitor  acted  as  military  medical  liaison  to  the  IRB  and  NPS  chain  of  command  in  the 
event  a  participant  was  injured.  Participants  were  required  to  perform  all  safety 
procedures,  demonstrate  how  to  properly  load  and  reload  the  weapon,  and  perform 
immediate  actions  in  the  event  of  a  weapon  malfunction  prior  to  the  live  fire  event.  One 
Range  Safety  Officer  (RSO)  was  present  and  controlling  the  range  during  the  live  fire 
event  and  provided  a  safety  brief  to  participants  prior  to  the  event. 
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D.  MEASURES 


1.  Demographics 

A  general  demographics  survey  was  administered,  which  contained  questions 
about  such  topics  as  age,  service,  pay  grade,  and  gender  (see  Table  1). 

2.  Marksmanship  Experience  Survey 

A  survey  was  administered  that  asked  questions  regarding  participants’  previous 
marksmanship  experience  (see  Table  2). 

E.  ISMT  SCENARIOS 

The  ISMT  system  used  for  this  research  was  a  USMC  version  and  contained  only 
USMC-approved  scenarios.  The  three  ISMT  scenarios  (zeroing,  unlimited  practice,  and 
qualification  NHQC  firing  sequence)  that  were  used  for  this  research  were  designed  by 
Tommy  Getty  for  a  previous  study.  For  more  information  on  the  design  and  settings  used 
for  each  scenario,  see  Getty  (2008). 

1.  Zeroing  Scenario 

First,  the  zeroing  scenario  was  used  to  calibrate  the  M9  prior  to  the  unlimited 
practice  and  NHQC  firing  sequence  scenarios  for  both  the  control  and  simulation  groups. 
This  scenario  had  no  time  limit  and  provided  only  three  rounds  in  the  magazine. 
Participants  were  required  to  shoot  three  rounds  center  target  at  three  yards  in  order  to 
calibrate  the  M9  to  the  ISMT  system.  Making  sure  the  M9  was  properly  calibrated 
ensured  the  accuracy  of  the  weapon.  The  zeroing  scenario  was  also  used  to  complete 
operational  testing  of  the  M9  and  aided  in  troubleshooting  the  weapon.  No  data  was 
collected  from  this  scenario. 

2.  Unlimited  Practice  Scenario 

The  unlimited  practice  scenario  provided  each  participant  with  the  opportunity  to 
practice  firing  the  M9  with  unlimited  rounds  and  no  time  restrictions.  This  scenario 
consisted  of  Transtar  II  silhouette  targets  at  the  3-,  7-,  and  15-yard  lines,  simultaneously 
displayed.  The  participants  in  the  control  group  used  this  scenario  as  a  means  to  conduct 
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dry  fire.  To  facilitate  a  dry  fire  environment,  the  ISMT  unlimited  practice  screen  was 
displayed,  but  the  M9  was  disconnected  from  the  system  and  CO2  cylinder  so  there  was 
no  feedback  from  the  system.  The  only  feedback  provided  to  participants  in  the  control 
group  was  from  the  instructor.  For  participants  in  the  simulation  group,  the  M9  was 
connected  to  the  ISMT  system  to  provide  system  and  audio  feedback  of  shots  fired  and 
CO2  cylinder  to  simulate  recoil  for  the  weapon.  The  instructor’s  control  panel  was  turned 
toward  the  participant,  allowing  the  participant  to  see  the  system-generated  feedback 
provided  by  ISMT  that  is  available  for  instructors  to  evaluate  the  participant  when  using 
the  system.  The  information  provided  included  round  count,  score,  and  pictures  of  each 
target  with  markers  to  represent  where  the  shots  intersected  the  target.  In  addition  to 
receiving  immediate  feedback  from  ISMT,  the  simulation  group  also  received  feedback 
from  the  instructor.  The  M9  magazine  contained  only  six  rounds  for  the  unlimited 
scenario.  More  rounds  could  have  been  programmed  into  the  system;  however,  having 
only  six  rounds  required  the  participants  to  load  and  unload  magazines  more  frequently, 
providing  more  reload  practice.  For  both  groups,  the  instructor  ensured  that  the 
participant  was  lined  up  on  the  center  line  of  the  respective  target  at  which  the  participant 
was  aiming.  No  data  was  collected  from  this  scenario. 

3.  Navy  Handgun  Qualification  Course  Firing  Sequence 

The  last  scenario  used  was  the  NHQC  firing  sequence.  All  data  collection 
occurred  using  this  scenario.  The  NHQC  firing  sequence  scenario  consisted  of  three 
separate  Transtar  II  silhouette  targets,  displayed  one  at  a  time  and  synced  with  the 
recorded  instructions.  Each  target  would  appear  once  the  recorded  instructions  were 
complete  and  prior  to  the  first  shot  (see  Figure  1). 


15 


Figure  1 .  Transtar  II  Silhouette  Target 

After  the  last  cease  fire  instruction  played,  the  target  would  disappear.  The 
participant  was  given  two  magazines  with  six  rounds  in  each  magazine  for  the  3-  and 
7-yard  line  firing  sequences.  For  the  15-yard  line  firing  sequence,  the  participant  was 
given  two  magazines  with  12  rounds  in  each  magazine.  The  recorded  instructions  for  the 
NHQC  firing  sequence  were  recorded  verbatim  from  OPNAVINST  3591. IF  (Chief  of 
Naval  Operations,  2009).  For  more  information  on  voice  recorded  training  instructions, 
see  Getty  (2008).  Refer  to  Table  3  for  a  detailed  description  of  NHQC  firing  sequence, 
Figure  2  for  the  participant’s  view  of  the  simulated  range,  and  Figure  3  for  the 
instructor’s  control  panel  display  screen. 
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Table  3.  Navy  Handgun  Qualification  Course  Firing  Sequence 
Yard  Line  Rounds _ Sequence _ Remarks 


3 

12 

Draw  and  fire  2  rounds  in  4  seconds 

2  rounds  strong  hand  supported  position 

Draw  and  fire  2  rounds  in  4  seconds 

2  rounds  strong  hand  supported  position 

Draw  and  fire  2  rounds,  reload  6  rounds 
and  fire  2  rounds  in  10  seconds 

4  rounds  strong  hand  supported  position 

Draw  and  fire  4  rounds  in  8  seconds 

2  rounds  strong  hand  supported  position, 
2  rounds  weak  hand  supported  position 

7 

12 

Draw  and  fire  2  rounds  in  4  seconds 

2  rounds  strong  hand  supported  position 

Draw  and  fire  2  rounds  in  4  seconds 

2  rounds  strong  hand  supported  position 

Draw  and  fire  2  rounds,  reload  6  rounds 
and  fire  2  rounds  in  10  seconds 

4  rounds  strong  hand  supported  position 

Draw  and  fire  4  rounds  in  8  seconds 

2  rounds  strong  hand  supported  position, 
2  rounds  weak  hand  supported  position 

15 

24 

Draw  and  fire  2  rounds  in  4  seconds 

2  rounds  strong  hand  supported  position 

Draw  and  fire  2  rounds  in  4  seconds 

2  rounds  strong  hand  supported  position 

Draw  and  fire  4  rounds  in  8  seconds 

4  rounds  strong  hand  supported  position 

Draw  and  fire  4  rounds,  reload  12 
rounds  and  fire  4  rounds  in  10  seconds 

8  rounds  strong  hand  supported  position 

Draw  and  fire  8  rounds  in  20  seconds 

8  kneeling  position 

Figure  2.  Participant’s  View 
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Figure  3.  Instructor  Control  Panel 


F.  PROCEDURES 

All  participants  were  active  duty  U.S.  military  members  and  either  NPS  students 
or  faculty.  Recruitment  was  conducted  via  e-mail.  Prior  to  a  participant’s  first  session,  a 
demographic  survey  was  sent  in  a  separate  e-mail  with  instructions  to  bring  the 
completed  survey  to  the  participant’s  first  session.  The  demographics  survey  contained 
questions  about  the  participants’  general  characteristics,  marksmanship  skill  level,  and 
marksmanship  experience. 

Participants  were  randomly  assigned  into  four  groups.  Training  time  blocks  of 
50  minutes  each  were  pre-designated,  based  on  the  authors’  schedules,  as  available 
training  time  sessions.  Time  blocks  were  sent  to  each  participant  and  the  participant  was 
instructed  to  sign  up  for  the  first  session,  with  the  second  session  as  an  automatic 
assignment  that  was  the  same  day  and  time  as  the  first  session,  but  exactly  one  week 
later.  The  authors  randomly  designated  time  blocks  as  either  control  group  or  simulation 
group  prior  to  participants  signing  up  for  training  sessions.  Participants  were  unaware  of 
which  group  they  were  assigned  to  until  they  arrived  for  their  second  training  session. 

A  baseline  of  their  current  level  of  marksmanship  performance  and  score  was 
recorded  via  the  ISMT  before  exposure  to  either  training  condition.  The  control  groups 
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received  SNMT  using  the  ISMT  unlimited  practice  scenario  screen  to  provide  visual 
Transtar  II  silhouette  target  s  for  the  training  session,  but  received  no  system  feedback 
during  the  training  session,  and  the  simulation  groups  received  simulation  training  using 
the  ISMT  unlimited  practice  scenario  with  full  system  feedback  during  the  training 
session.  One  control  group  and  simulation  group  received  no  additional  training  for  two 
weeks,  while  the  other  control  group  and  simulation  group  received  no  additional  training 
for  four  weeks.  After  their  respective  time  lapse,  all  groups  conducted  a  final  live  fire 
NHQC  firing  sequence  to  determine  retention  of  marksmanship  knowledge. 

Groups  were  staggered  so  only  four  participants  completed  the  live  fire  events  at  a 
time.  During  all  NHQC  firing  sequences,  data  regarding  participants’  MPI  and 
performance  score  were  collected.  The  dependent  variables  were  MPI  of  group  shots  and 
scores  from  the  NHQC  firing  sequence.  The  participants  were  not  video  or  audio 
recorded  at  any  time.  Appropriate  statistical  analyses  on  the  performance  scores  and  MPI 
were  used  to  test  the  hypotheses. 

During  the  course  of  this  research,  several  controls  were  in  place  to  ensure 
standardization.  In  the  scheduling  phase  of  this  research,  when  participants  signed  up  for 
the  first  training  session  they  needed  to  ensure  they  were  available  to  complete  their 
second  training  session  exactly  one  week  later.  Each  training  session  was  limited  to 
50  minutes  to  ensure  every  participant  received  an  equal  amount  of  training  time.  All 
participants  used  the  isosceles  standing  position  for  the  first  36  rounds  fired  and  the 
isosceles  kneeling  position  for  the  last  12  rounds  fired  to  mitigate  any  confounds 
associated  with  various  firing  positions  and  maximize  consistency  (see  Figures  4  and  5). 
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Figure  4. 


Isosceles  Standing  Position  (From  Headq 
United  States  Marine  Corps,  2003 


2! 


$ 


Figure  5.  Isosceles  High  Kneeling  Position  (From  Headquarters, 

United  States  Marine  Corps,  2003) 

The  NHQC  firing  sequence  was  used  for  all  data  collection  shooting  events.  Each 
participant  shot  from  the  same  location  marked  by  blue  masking  tape  on  the  floor  of  the 
ISMT  trailer  to  indicate  the  center  line  to  help  participants  center  their  bodies  on  the 
target.  Every  participant  used  the  same  style  holster  for  both  simulation  and  live  fire 
events,  which  consisted  of  a  standard  issue  police  officer  holster  with  weapon  locking 
mechanism.  Each  participant  was  instructed  to  place  the  weapon  in  the  holster  in  such  a 
way  that  the  weapon  rested  just  above  the  locking  mechanism  threshold,  thus  ensuring 
the  weapon  never  locked  while  in  the  holster.  The  holster  belt  was  fully  adjustable, 
thereby  allowing  each  participant  to  adjust  as  needed  to  ensure  a  proper  fit.  All 
instructions  were  recorded  verbatim  from  OPNAVINST  3591. IF  (Chief  of  Naval 
Operations,  2009)  to  ensure  standardization  and  consistency. 

Before  participants  arrived  for  a  session,  the  authors  ensured  the  ISMT  was  ready. 
First,  a  ready  line  and  a  firing  line  were  established  in  accordance  with  OPNAVINST 
3591.  IF  (Chief  of  Naval  Operations,  2009)  and  marked  in  ISMT  by  placing  masking  tape 
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on  the  deck  at  the  appropriate  distances  from  the  projector  screen.  General  safety  rules 
were  posted  next  to  the  firing  line  and  participants  were  instructed  to  read  them  prior  to 
commencing  training.  The  authors  utilized  a  personal  laptop  with  audio  files  containing 
recorded  instructions.  The  laptop  was  connected  to  speakers  mounted  in  various 
locations  inside  the  ISMT  lab.  The  authors  ensured  the  ISMT  system  was  calibrated 
prior  to  each  participant’s  arrival.  Following  system  calibration,  the  M9  was  registered 
and  calibrated  to  the  ISMT.  The  registration  process  consisted  of  the  instructor  firing  one 
round  at  the  screen  when  prompted  by  the  ISMT.  The  laser-reading  camera  would  read 
the  individual  pulse-coded  laser  for  that  particular  weapon  and  register  it  in  the  ISMT 
system.  The  weapon  calibration  process  is  mentioned  above  in  the  zeroing  scenario. 
During  the  first  training  session,  each  participant  provided  a  unique,  four-digit  identifier 
that  was  used  to  correlate  performance  data  to  that  participant  through  all  training  and 
evaluation  sessions. 

Prior  to  beginning  the  baseline  training,  every  participant  received  a  safety  brief 
in  addition  to  an  overview  of  the  procedures  used  in  this  part  of  the  training.  The  safety 
brief  was  conducted  in  accordance  with  OPNAVINST  3591. IF  (Chief  of  Naval 
Operations,  2009)  to  ensure  all  safety  requirements  were  met.  Baseline  training  ensured 
that  every  participant  began  with  a  basic  level  of  knowledge  on  how  to  properly  handle 
and  fire  the  M9,  which  included  the  proper  use  of  the  M9  and  specific  weapon  commands 
for  the  experiment  (Getty,  2008).  Table  4  provides  a  detailed  description  of  the  baseline 
training  topics. 
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Table  4.  Baseline  Training  Program  (From  Getty,  2008) 


Baseline  Training 

General  Range  Safety  Rules 
General  Safety  Rules 
Weapon  Conditions 
Weapon  Commands 
Pistol  Safety  Rules 
Ready  Line 
Firing  Line 
Range  Operations 
Training  Time  Out 
Loading  the  Pistol 
Making  the  Pistol  Ready 
Fire 

Cease  Fire 

Unload,  Showing  the  Pistol  Clear 
Dry  Reload 
Isosceles  only 
Isosceles  Standing  Position 
Isosceles  High  Kneeling  Position 


Immediately  following  baseline  training,  a  baseline  simulation  shooting  event  was 
administered  to  each  participant  using  the  NHQC  firing  sequence.  The  purpose  of  this 
shooting  event  was  to  establish  baseline  performance  of  for  each  participant  to  compare 
to  their  post-training  performance. 

The  next  training  session  consisted  of  either  control  group  training  or  simulation 
group  training.  No  participant  was  aware  of  which  group  they  were  in  until  they  showed 
up  for  the  second  training  session.  The  control  group  training  for  this  thesis  followed  the 
control  group  training  that  Getty  (2008)  used: 

The  control  group  training  program  was  created  by  using  enclosure  two  of 
the  OPNAVINST  3591. IF  (Chief  of  Naval  Operations,  2009),  chapter  two 
of  the  NTRP3-07.2.2  (U.S.  Fleet  Forces  Command,  2003)  and  chapter 
three  of  the  MCRP  3-01B  (Headquarters,  United  States  Marine  Corps, 

2003).  The  participants  were  allowed  to  handle  the  M9  Berretta  and 
follow  along  with  the  recorded  lessons.  It  was  important  that  the 
participants  were  afforded  the  opportunity  to  practice  without  any  time 
constraints.  During  the  lecture,  the  participants  focused  on  slow,  smooth 
and  methodical  motions  when  drawing  and  dry  firing  the  pistol.  The 
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participants  in  the  control  group  received  18  minutes  of  recorded  lecture. 

(p.  10) 

The  unlimited  practice  scenario  was  used  for  control  group  training,  which 
displayed  three  Transtar  II  silhouette  target  s  simultaneously  for  the  3-,  7-,  and  15-yard 
lines.  During  this  training,  the  M9  was  disconnected  from  the  ISMT.  The  only 
immediate  feedback  available  to  control  group  participants  was  from  the  instructor. 
Table  5  provides  a  detailed  description  of  the  control  group  training  topics. 
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Table  5. 


Control  Group,  Marksmanship  Fundamental  Training  Program 
(From  Getty,  2008) 


Control  Group  Training 

Dry  Fire  Safety  Training 
Drawing  Process 
Grab 
Draw 
Smack 
Look  Squeez 
Holstering 

Introduction  to  Fundamentals 
Aiming 

Sight  Alignment 
Establishing  Sight  Alignment 
Grip  with  respect  to  (WRT)  Aiming 
Controlled  Muscular  Tension 
Sight  Picture 

Relationship  Betweent  the  Eye  and  the  Sights 
Trigger  Control 

Sight  Alignment  and  Trigger  Control 
Grip  WRT  Trigger  Control 
Trigger  Finger  Placement 
Uninterrupted  Trigger  Control 
Interrupted  Trigger  Control 
Breath  Control 
Begin  Dry  Fire  Training 
Single  Action 

Five  to  Ten  Times  Repitition 
Start  Adding  Time  Limits 
Draw  Sweep  Safety  Dry  Fire 


Immediately  following  control  group  training,  the  M9  was  reconnected  to  the 
ISMT  and  the  post-training  shooting  event  was  administered  using  the  NHQC  firing 
sequence.  The  purpose  of  this  shooting  event  was  to  compare  post-training  performance 
to  the  participants’  baseline  performance. 

The  simulation  group  training  was  similar  to  the  control  group  except  that  instead 
of  dry  fire  training,  the  M9  was  connected  to  the  ISMT  system  to  provide  system  and 
audio  feedback  of  shots  fired  and  connected  to  the  CO2  cylinder  to  provide  simulated 
recoil  for  the  weapon.  The  instructor’s  control  panel  was  turned  toward  the  participant, 
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providing  the  same  system-generated  feedback  that  is  provided  for  the  instructor  to 
evaluate  the  participant.  The  information  provided  included  round  count,  score,  and 
pictures  of  each  target  with  markers  to  represent  where  the  shots  intersected  the  target. 
The  training  duration  was  16  minutes;  two  minutes  shorter  than  control  group  training. 
The  shorter  training  time  is  due  to  not  having  to  provide  dry  fire  instructions,  but  was 
compensated  by  showing  simulation  group  participants  trace  profiles  from  their  baseline 
qualification  shooting  event.  The  simulation  group  participants  also  utilized  the 
unlimited  practice  scenario.  Table  6  provides  a  detailed  description  of  the  simulation 
group  training. 


Table  6.  Simulation  Group  Training,  Marksmanship  Fundamental  Training  Program 

(From  Getty,  2008) 

Simulation  Group  Training 

Simulation  Safety  Training 
Drawing  Process 
Grab 
Draw 
Smack 
Look  Squeez 
Holstering 

Introduction  to  Fundamentals 
Aiming 

Sight  Alignment 
Establishing  Sight  Alignment 
Grip  with  respect  to  (WRT)  Aiming 
Controlled  MuscularTension 
Sight  Picture 

Relationship  Betweent  the  Eye  and  the  Sights 
Trigger  Control 

Sight  Alignment  and  Trigger  Control 
Grip  WRT  Trigger  Control 
Trigger  Finger  Placement 
Uninterrupted  Trigger  Control 
Interrupted  Trigger  Control 
Breath  Control 
Begin  Simulation  Training 
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Immediately  following  simulation  group  training,  a  post-training  shooting  event 
was  administered  using  the  NHQC  firing  sequence.  The  purpose  of  this  shooting  event 
was  to  compare  post-training  performance  to  the  participants’  baseline  performance. 
Figure  6  is  a  visual  representation  of  the  research. 


Week  1 

RECRUIT  PARTICIPANTS 

Week  2 

ISMT  BASEL  ENT 
QUALIFICATION 

ISMT  BASELINE 
QUALIFICATION 

i 

Week  3 

ISMT  TRAINING  AND  POST 
TRAINING  QUALIFICATION 

STANDARD  TRAINING  AND 
POST  TRAINING 
QUALIFICATION 

_ 

Week  4 

NO  TRAINING 

ISMT  BASELINE 
QUALIFICATION 

NO  TRAINING 

ISMT  BASELINE 
QUALIFICATION 

i 

Week  5 

NO  TRAINING 

ISMT  TRAINING  AND  POST 
TRAINING  QUALIFICATION 

NO  TRAINING 

STANDARD  TRAINING  AND 
POST  TRAINING 
QUALIFICATION 

i 

Week  6 

NO  TRAINING 

NO  TRAINING 

NO  TRAINING 

i 

Week  7 

NO  TRAINING 

NO  TRAINING 

NO  TRAINING 

NO  TRAINING 

 1 

Week  8 

FINAL  LIVE  FIRE 
QUALIFICATION 

FINAL  LIVE  FIRE 
QUALIFICATION 

FINAL  LIVE  FIRE 
QUALIFICATION 

FINAL  LIM  FIRE 
QUALIFICATION 

Figure  6.  Research  Design 
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A  detailed  description  of  the  research  design  is  as  follows: 

Week  1: 

All  Participants:  Thirty-four  participants  were  recruited.  A  demographic  survey  was 
administered  prior  to  the  beginning  of  the  experiment  to  gather  information  about 
previous  marksmanship  experience.  Questions  included  general  characteristics, 
marksmanship  skill  level,  and  weapon  experience. 

Week  2: 

First  Control  Group:  Baseline  ISMT  event  consisting  of  the  NHQC  firing  sequence  was 
administered  to  establish  their  current  level  of  marksmanship. 

First  Simulation  Group:  Baseline  ISMT  event  consisting  of  the  NHQC  firing  sequence 
was  administered  to  establish  their  current  level  of  marksmanship. 

Week  3: 

First  Control  Group:  Received  SNMT,  which  involved  demonstrating  proper  techniques 
for  marksmanship  including  proper  stance,  proper  breathing  control,  and  conducting  dry 
fire  with  a  simulated  M9.  These  participants  had  the  opportunity  to  fire  practice  shots  at 
simulated  Transtar  II  silhouette  targets,  but  no  system  feedback  was  provided  from 
ISMT.  Once  training  was  complete,  a  post-training  ISMT  event  consisting  of  the  NHQC 
firing  sequence  was  administered  to  establish  any  differences  between  post-training 
and  baseline. 

First  Simulation  Group:  Received  simulation-based  training  in  the  ISMT,  which 
involved  demonstrating  proper  techniques  for  marksmanship  including  proper  stance, 
proper  breathing  control,  and  conducting  dry  fire  with  a  simulated  M9.  These 
participants  also  had  the  opportunity  to  fire  practice  shots  at  simulated  Transtar  II 
silhouette  targets,  and  analyze  their  trace  profiles  and  grouping  of  shots.  The  analysis 
component  was  the  difference  in  training  between  the  control  and  simulation  groups. 
Once  training  was  complete,  a  post-training  ISMT  event  consisting  of  the  NHQC  firing 
sequence  was  administered  to  establish  any  differences  between  post-training 
and  baseline. 
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Week  4: 

First  Control  Group:  Received  no  additional  training. 

First  Simulation  Group:  Received  no  additional  training. 

Second  Control  Group:  Baseline  ISMT  event  consisting  of  the  NHQC  firing  sequence 
was  administered  to  establish  their  current  level  of  marksmanship. 

Second  Simulation  Group:  Baseline  ISMT  event  consisting  of  the  NHQC  firing  sequence 
was  administered  to  establish  their  current  level  of  marksmanship. 

Weeks  5: 

First  Control  Group:  Received  no  additional  training. 

First  Simulation  Group:  Received  no  additional  training. 

Second  Control  Group:  Received  SNMT,  which  involved  demonstrating  proper 
techniques  for  marksmanship  including  proper  stance,  proper  breathing  control,  and 
conducting  dry  fire  with  a  simulated  M9.  These  participants  had  the  opportunity  to  fire 
practice  shots  at  simulated  Transtar  II  silhouette  targets,  but  no  system  feedback  was 
provided  from  ISMT.  Once  training  was  complete,  a  post-training  ISMT  event  consisting 
of  the  NHQC  firing  sequence  was  administered  to  establish  any  differences  between 
post-training  and  baseline. 

Second  Simulation  Group:  Received  simulation-based  training  in  the  ISMT,  which 
involved  demonstrating  proper  techniques  for  marksmanship  including  proper  stance, 
proper  breathing  control,  and  conducting  dry  fire  with  a  simulated  M9.  These 
participants  also  had  the  opportunity  to  fire  practice  shots  at  simulated  Transtar  II 
silhouette  targets,  and  analyze  their  trace  profiles  and  grouping  of  shots.  The  analysis 
component  was  the  difference  in  training  between  the  control  and  simulation  groups. 
Once  training  was  complete,  a  post-training  ISMT  event  consisting  of  the  NHQC  firing 
sequence  was  administered  to  establish  any  differences  between  post-training 
and  baseline. 

Weeks  6: 

All  Participants:  Received  no  additional  training. 

Week  7: 

All  Participants:  Received  no  additional  training. 
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Week  8: 

All  Participants:  Conducted  live  fire  events  at  Laguna  Seca  Range  on  the  NHQC  firing 
sequence.  Prior  to  beginning  the  live  fire  events,  the  RSO  provided  a  safety  brief  to 
participants.  This  shooting  event  determined  how  much  carryover  each  participant  had  of 
marksmanship  skills  from  simulation  to  the  live  fire  event  with  a  two-  or  four-week  gap. 
This  was  the  final  event  for  all  groups. 
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III.  RESULTS 


A.  DATA  PREPARATION  AND  STATISTICAL  METHODS 

Demographics  survey  data  was  transcribed  from  paper  surveys  and  recorded  in  an 
Excel  spreadsheet.  It  was  analyzed  using  two -proportion  Z-tests,  y\  and  two-sample 
Mcsts  with  the  data  analysis  tool  package  in  Excel  2007. 

All  experimental  data  sets  from  ISMT  were  transcribed  from  computer-generated 
data  screens  into  an  Excel  spreadsheet,  a  picture  of  each  data  screen  in  ISMT  was  taken 
for  back  up,  and  a  video  of  each  trace  profile  was  recorded  for  each  ISMT  event.  The 
live  fire  event  paper  targets  were  graded  without  knowledge  of  which  group  the  shooter 
was  assigned.  All  live  fire  data  sets  were  recorded  in  an  Excel  spreadsheet. 
Experimental  data  sets  were  analyzed  using  two-sample  Mcsts,  paired  Mcsts,  and 
Wilcoxon  Rank  Sum  Tests.  Statistical  calculations  for  experimental  data  sets  were 
accomplished  using  the  data  analysis  tool  package  in  Excel  2007  or  JMP  9. 

The  two  main  variables  for  hypothesis  testing  were  MPI  and  score.  MPI  is 
defined  in  Joint  Publication  1-02  as  “[t]he  point  whose  coordinates  are  the  arithmetic 
means  of  the  coordinates  of  the  separate  points  of  impact/burst  of  a  finite  number  of 
projectiles  fired  or  released  at  the  same  aiming  point  under  a  given  set  of  conditions” 
(Joint  Chiefs  of  Staff,  2010,  p.  207).  MPI  is  measured  in  centimeters,  with  lower  values 
indicating  more  accurate  performance.  Scores  were  assessed  based  on  OPNAVINST 
3591. IF  (Chief  of  Naval  Operations,  2009).  As  stated  in  Chapter  II,  each  shooter 
participated  in  three  shooting  events:  an  initial  baseline  shooting  event  in  ISMT,  a 
post-training  shooting  event  in  ISMT,  and  a  final  live  fire  shooting  event  at  Laguna  Seca 
Peace  Corps  Range.  Each  shooting  event  consisted  of  three  distances:  3-,  7-,  and 
15-yards.  Scores  and  MPI  for  each  distance  were  recorded  for  all  shooting  events.  ISMT 
provided  the  MPI  for  each  target  simulation  as  well  as  the  average  MPI  across  the  three 
distances.  For  each  paper  target  from  the  live  fire  event,  the  MPI  for  each  distance  from 
the  target  was  calculated  by  measuring  the  distance  between  the  shot  and  the  center  of  the 
target.  The  average  MPI  for  the  live  fire  event  was  then  calculated  in  Excel.  To  reduce 
Type  I  error,  the  averaged  MPI  across  the  three  distances  from  the  target  for  each 


31 


shooting  event  was  used  for  hypothesis  testing.  Scoring  for  each  shooting  event  was 
graded  in  accordance  with  the  NHQC  instruction,  based  on  each  shooter’s  performance, 
with  higher  scores  indicating  better  performance.  There  was  a  miscalculation  in  the 
user-defined  ISMT  scenario  for  the  average  group  shot,  so  that  variable  was  not  analyzed 
in  this  research. 

For  the  averaged  MPIs  and  score,  difference  scores  were  calculated  on  the 
following  data  sets:  baseline  to  post-training  performance,  baseline  to  live  fire 
performance,  and  post-training  to  live  fire  performance.  The  difference  scores  were  used 
to  assess  changes  in  performance  over  time  (i.e.,  across  the  three  shooting  events)  to 
determine  whether  any  significant  improvement  or  degradation  occurred  within  each 
group.  These  difference  scores  were  analyzed  using  two-sample  /-tests  to  determine 
whether  or  not  the  simulation  group  showed  greater  levels  of  improvement  than  the 
control  group.  These  difference  scores  also  were  used  to  test  the  hypothesis  regarding 
change  in  performance  between  participants,  with  a  two-week  gap  and  those  with  a 
four-week  gap,  from  post-training  to  live  fire.  The  control  and  simulation  group  sizes  for 
this  research  were  too  small  to  effectively  test  hypothesis  two.  Therefore,  the  group  with 
a  two-week  gap  between  post-training  and  live  fire  had  seven  participants  in  the 
simulation  group  and  eight  in  the  control  group;  and  the  group  with  a  four-week  gap 
between  post-training  and  live  fire  had  five  participants  in  the  simulation  group  and  seven 
in  the  control  group. 

A  one-tailed  alpha  of  0.05  was  used  for  all  hypothesis  testing. 

A  non-parametric  Wilcoxon  Rank  Sum  Test  was  used  on  all  difference  scores, 
difference  MPI  measurements,  and  two-week  to  four-week  gap  comparisons  due  to 
violation  of  the  normality  assumption.  All  Wilcoxon  Rank  Sum  Tests  were  performed  in 
JMP  9.  The  results  from  all  non-parametric  tests  had  the  same  pattern  as  the  two-sample 
/-tests.  Therefore,  only  two-sample  /-test  results  are  described  for  ease  of  interpretation. 
An  F-test  was  calculated  on  each  data  set  to  check  the  equal  variance  assumption  prior  to 
conducting  a  two-sample  /-test  for  each  comparison.  Based  on  the  results  of  the  F-test, 
the  appropriate  two-sample  /-test  was  selected.  All  F-tests  and  /-tests  were  performed 
in  Excel. 
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B.  PRELIMINARY  ANALYSES 


1.  Preliminary  Analysis  for  Hypothesis  One 

Before  conducting  testing  for  hypothesis  one,  scores  and  MPI  measurements  were 
evaluated  on  the  following  data  sets  to  determine  if  there  were  differences  in  performance 
levels  during  each  evaluated  shooting  event  between  the  control  group  and  the  simulation 
group:  (1)  at  baseline,  (2)  at  post-training,  and  (3)  at  live  fire.  First,  the  baseline 
performance  between  groups  at  baseline  was  compared  to  determine  if  the  groups  started 
at  the  same  level  of  marksmanship  performance.  Then,  a  within-group  analysis  was 
conducted  to  examine  whether  each  group  showed  significant  improvement 
from  baseline. 


a.  Scores 

The  P-values  indicated  no  significant  baseline  difference  in  scores 
between  the  control  and  simulation  group  in  any  of  the  shooting  events.  Comparisons  of 
the  baseline  shooting  events  suggest  that  the  simulation  group  (210.33  ±  21.94  points) 
was  performing  slightly  better  than  the  control  group  (201.44  ±21.25  points)  for  score;  a 
point  that  will  be  further  discussed  in  Chapter  IV.  There  also  was  a  trend  for  the 
simulation  group  to  outperform  the  control  group  in  post-training  (t( 29)  =  -1.38, 
p  =  0.089).  Both  groups  showed  significant  improvement  from  baseline.  The  P-values 
suggest  that  the  control  group  showed  significant  improvement  in  score  from  baseline  to 
post-training  (mean  difference  =  26.88  ±  16.16  points),  (/(l  5)  =  6.65,  p  <  0.0001),  and 
from  baseline  to  live  fire  (mean  difference  =  15.81  ±  15.48  points),  ( t( 1 5 )  =  4.09, 
p  =  0.0005).  The  simulation  group  also  showed  significant  improvement  in  score  from 
baseline  to  post-training  (mean  difference  =  22.40  ±  19.98  points),  (/( 1 4)  =  4.34, 
p  =  0.0003),  and  from  baseline  to  live  fire  (mean  difference  =  7.83  ±  13.69  points), 
(t(ll)  =  1.98,  p  =  0.0365). 

b.  MPI 

The  P-values  indicated  no  significant  baseline  difference  in  MPI  between 
the  control  and  simulation  group  at  baseline.  However,  the  simulation  group 
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outperformed  the  control  group  in  MPI  at  post-training  (t( 29)  =  2.52,  p  =  0.009)  and  live 
fire  (t(26)  =  -0.77,  p  =  0.020).  These  results  are  consistent  with  the  within-group 
analyses.  The  control  group  showed  significant  degradation  in  MPI  from  post-training  to 
live  fire  (mean  degradation  =  2.79  ±  3.26  cm),  (t(15)  =  2.38,  p  =  0.0154),  whereas  the 
simulation  group  was  less  likely  to  show  degraded  MPI  performance  from  post-training 
to  live  fire  (mean  degradation  =  2.51  ±  3.05  cm),  (t(ll)  =  1.60,  p  =  0.0685). 

Means  and  standard  deviations  for  scores  and  MPI  at  each  shooting  event 
by  group  are  shown  in  Table  7. 


Table  7.  Comparison  of  Control  and  Simulation  Groups  at  Each  Shooting  Event 


Baseline 

I 

’ost-Training 

Live  Fire 

Control 

(n=16) 

Simulation 

(n=15) 

Control 

(n=16) 

Simulation 

(n=15) 

Control 
(n  =  16) 

Simulation 

(n=12) 

Score 

Mean 

201.44 

210.33 

Mean 

228.31 

232.73 

Mean 

217.25 

222.67 

sd 

21.25 

21.94 

sd 

8.90 

8.90 

sd 

20.55 

14.90 

MPI 

Mean 

5.28 

5.25 

Mean 

4.13 

2.61 

Mean 

6.85 

4.92 

sd 

2.37 

2.05 

sd 

1.95 

1.31 

sd 

2.86 

2.18 

Bar  charts  with  standard  error  bars  are  provided  for  scores  (Figure  7)  and 
MPI  (Figure  8)  for  a  visual  representation  of  each  shooting  event. 
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240.00 

230.00 

220.00 

210.00 

200.00 

190.00 

180.00 


Control 

(n=16) 

Simulation 

(n=15) 

Control 

(n=16) 

Simulation 

(n=15) 

Control 

(n=16) 

Simulation 

(n=12) 

Baseline 

Post-Training 

Live  Fire 

Figure  7.  Bar  Chart  of  Score  Comparison  of  Control  and  Simulation 

34 


MPI 


Control 

(n=16) 

Simulation 

(n=15) 

Control 

(n=16) 

Simulation 
(n=  15) 

Control 

(n=16) 

Simulation 

(n=12) 

Baseline 

Post-Training 

Live  Fire 

Figure  8.  Bar  Chart  of  MPI  Comparison  of  Control  and  Simulation  Groups 


2.  Preliminary  Analysis  for  Hypothesis  Two 

Before  conducting  testing  for  hypothesis  two,  scores  and  MPI  measurements  were 
evaluated  on  the  following  data  sets  to  determine  if  there  was  a  difference  in  performance 
levels  during  each  evaluated  shooting  event  between  participants  with  a  two-week  gap 
from  post-training  to  live  fire  (two-week  gap  group)  and  participants  with  a  four-week 
gap  from  post-training  to  live  fire  (four- week  gap  group):  (1)  at  baseline,  (2)  at 
post-training,  and  (3)  at  live  fire. 

a.  Scores 

While  the  P-values  indicate  no  significant  baseline  difference  in  scores 
between  the  two-week  gap  group  and  the  four-week  gap  group  in  any  of  the  shooting 
events,  the  post-training  scores  trend  toward  significance  in  which  the  two-week  gap 
group  outperformed  the  four- week  gap  group  (t( 23)  =  1.56,  p  =  0.067).  The  paired  t- test 
indicated  that  both  the  two-  and  four-week  gap  groups  showed  significant  improvement 
in  score  from  baseline  to  live  fire  (two-week  gap:  9.93  ±  12.19  points),  (t(14)  =  3.16, 
p  =  0.0035;  four-week  gap:  15.23  +  17.84  points),  (t(12)  =  3.08,  p  =  0.0048). 
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b. 


MPI 


There  was  a  trend  for  the  two-week  gap  group  to  outperform  the 
four-week  gap  group  at  baseline  (t( 29)  =  1.38,  p  =  0.089)  and  post-training 
(t( 29)  =  -1.65,  p  =  0.055)  on  MPI.  However,  the  two-week  gap  group  actually  showed 
more  degradation  in  MPI  from  post-training  to  live  fire  (mean  degradation  =  3.41  ± 
3.10  cm),  (t(  14)  =  2.85,  p  =  0.0064)  than  the  four- week  group  (mean  degradation  =  2.15 
±  3.21  cm),  (t(  12)  =  1.28,  p  =  0.1117). 

Means  and  standard  deviations  for  scores  and  MPI  at  each  shooting  event 
by  time  gap  are  shown  in  Table  8. 


Table  8.  Comparison  of  Two-  and  Four-Week  Time  Gaps  Between 
Post-Training  and  Live  Fire  at  Each  Shooting  Event 


Baseline 

Post-Training 

Live  Fire 

2  Weeks 
(n=16) 

4  Weeks 
(71=75) 

2  Weeks 
(n—16) 

4  Weeks 
(n=15) 

2  Weeks 
(n=15) 

4  Weeks 
(n=13) 

Score 

Mean 

207.94 

203.40 

Mean 

232.88 

227.87 

Mean 

219.27 

219.92 

sd 

21.18 

22.73 

sd 

6.56 

10.72 

sd 

17.60 

19.66 

MPI 

Mean 

5.78 

4.71 

Mean 

2.89 

3.93 

Mean 

6.32 

6.09 

sd 

1.98 

2.32 

sd 

1.49 

2.02 

sd 

2.89 

3.04 

Bar  charts  with  standard  error  bars  are  provided  for  scores  (Figure  9)  and 
MPI  (Figure  10)  for  a  visual  representation  of  each  shooting  event. 
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Score 


2  Weeks 
(n=16) 

4  Weeks 
(n=15) 

2  Weeks 
(n=16) 

4  Weeks 
(n=15) 

2  Weeks 
(n=15) 

4  Weeks 
(n=13) 

Baseline 

Post-Training 

Live  Fire 

Figure  9.  Bar  Chart  of  Score  Two-  and  Four-Week  Time  Gaps  Between 

Post-Training  and  Live  Fire 


MPI 


2  Weeks 
(n=16) 

4  Weeks 
(n=15) 

2  Weeks 
(n=16) 

4  Weeks 
(n=15) 

2  Weeks 
(n=15) 

4  Weeks 
(n=13) 

Baseline 

Post-Training 

Live  Fire 

Figure  10.  Bar  Chart  of  MPI  Two-  and  Four-Week  Time  Gaps  Between 

Post-Training  and  Live  Fire 


C.  HYPOTHESIS  TESTING  AND  ANALYSIS 
1.  Hypothesis  One 

HOp  There  will  be  no  group  differences  in  the  participants’  MPI  of  shots  and 
scores  in  the  NHQC  firing  sequence  from  baseline  to  live  fire. 
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HAj:  Participants  who  receive  simulation-based  training  in  ISMT  will  have  a 
greater  improvement  in  MPI  of  shots  and  scores  in  the  NHQC  firing  sequence  when 
comparing  the  difference  between  baseline  and  live  fire  than  participants  who 
receive  SNMT. 

Change  in  scores  and  MPI  measurements  were  evaluated  on  the  following  data 
sets  to  assess  changes  in  performance  between  the  control  group  and  the  simulation 
group:  (1)  between  baseline  and  post-training  events,  (2)  between  post-training  and  live 
fire  events,  and  (3)  between  baseline  and  live  fire  events. 

a.  Scores 

No  significant  group  differences  were  found  in  assessing  changes  in  score 
performance  in  any  shooting  event  comparison.  However,  the  control  group  displayed  a 
trend  for  greater  improvement  than  the  simulation  group  from  baseline  to  live  fire 
0(26)  =  1.42,  p  =  0.084). 

b.  MPI 

The  P-values  indicate  significant  differences  in  change  in  MPI 
performance  from  baseline  to  live  fire,  where  the  simulation  group  showed  essentially  no 
degradation  from  baseline  to  live  fire;  however,  the  control  group  showed  degradation  in 
its  difference  MPI  measurements  0(26)  =  1-83,  p  =  0.039).  The  simulation  group  also 
demonstrated  a  trend  for  greater  improvement  than  the  control  group  from  baseline  to 
post-training  0(29)  =  1.58,  p  =  0.063). 

Means  and  standard  deviations  for  difference  scores  and  difference  MPI 
for  each  pair  of  shooting  events  by  group  are  shown  in  Table  9. 
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Table  9.  Comparison  of  Control  and  Simulation  Groups  for  Differences  in  Score 

and  MPI 


Difference 

(Baseline  to  Post-Training) 

Difference 

(Post-Training  to  Live 
Fire) 

Difference 

(Baseline  to  Live  Fire) 

Control 

(n=16) 

Simulation 

(n=15) 

Control 

(n-16) 

Simulation 
(n  =  12) 

Control 

(n=16) 

Simulation 
(n  =  12) 

Score 

26.88 

22.40 

Mean 

-11.06 

-11.50 

Mean 

15.81 

7.83 

16.16 

19.98 

sd 

14.62 

12.94 

Sd 

15.48 

13.69 

MPI 

-1.16 

-2.64 

Mean 

2.79 

2.51 

Mean 

1.75 

-0.01 

2.47 

2.77 

sd 

3.26 

3.05 

Sd 

2.99 

2.39 

2.  Results  of  Hypothesis  One  Testing  and  Analysis 

HOi  is  partially  rejected.  The  simulation  group  showed  significantly  less 
degradation  in  MPI  performance  than  the  control  group.  However,  there  was  a  trend 
towards  significance  for  the  control  group  to  show  greater  improvement  in  scores  than 
the  simulation  group. 

3.  Hypothesis  Two 

HOi:  Participants  in  each  group  will  be  equally  likely  to  maintain  their  MPI  of 
shots  and  scores  on  the  NHQC  two  weeks  and  four  weeks  after  the  training  day. 

HAi:  Participants  who  receive  simulation-based  training  in  ISMT  will  be  more 
likely  than  those  in  the  control  group  to  maintain  their  MPI  of  shots  and  scores  in  the 
NHQC  two  weeks  and  four  weeks  after  the  training  day. 

Changes  in  scores  and  MPI  measurements  were  evaluated  on  the  following  data 
sets  to  assess  maintenance  of  performance  between  the  two-week  gap  group  and  the 
four-week  gap  group:  (1)  between  post-training  and  live  fire  events  and  (2)  between 
baseline  and  live  fire  events. 

a.  Scores 

The  P-values  show  no  significant  differences  between  the  two-week  gap 
group  and  the  four-week  gap  group  for  scores  in  any  of  the  cases  tested.  Both  groups 
showed  a  pattern  of  performance  degradation  from  post-training  to  live  fire,  but  an 
overall  improvement  from  baseline  to  live  fire. 

39 


b. 


MPI 


There  were  no  significant  differences  found  between  the  two-week  gap 
group  and  the  four-week  gap  group  from  the  baseline  to  live  fire  or  from  the  post-training 
to  live  fire  comparisons.  Both  groups  showed  degradation  in  MPI  performance  over  the 
retention  interval. 

Means  and  standard  deviations  difference  scores  and  difference  MPI  for 
each  pair  of  shooting  events  by  time  gap  are  shown  in  Table  10. 


Table  10.  Comparison  of  Two-  and  Four-Week  Time  Gaps  Between 
Post-Training  and  Live  Fire  for  Differences  in  Score  and  MPI 


Difference 

(Baseline  to  Post-Training) 

Difference 

(Post-Training  to  Live  Fire) 

Difference 

(Baseline  to  Live  Fire) 

2  Weeks 
(n=16) 

4  Weeks 
(n=15) 

2  Weeks 
(n=15) 

4  Weeks 
(n=13) 

2  Weeks 
(n=15) 

4  Weeks 
(n=13) 

Score 

mm 

24.47 

Mean 

-13.40 

-8.77 

Mean 

9.93 

15.23 

m 

14.62 

sd 

15.38 

11.50 

sd 

12.19 

17.84 

MPI 

-2.90 

-0.78 

Mean 

2.15 

Mean 

0.69 

1.53 

2.76 

2.18 

sd 

3.21 

sd 

322 

2.38 

4.  Results  of  Hypothesis  One  Testing  and  Analysis 

HOo  is  retained.  The  P-values  show  no  significant  differences  between  the 
two-  and  four-week  gap  groups  for  scores  or  MPI  in  any  of  the  cases  tested. 

D.  EXPLORATORY  ANALYSES 

As  noted  above,  each  shooter  participated  in  an  initial  baseline  simulation 
shooting  event,  a  post-training  simulation  shooting  event,  and  a  final  live  fire  shooting 
event.  For  all  shooting  events,  the  shot  group  diameter  was  also  recorded.  Targets  were 
changed  out  at  each  firing  distance  (3,  7,  and  15  yards)  in  the  live  fire  shooting  event  to 
facilitate  comparisons  with  the  data  collection  from  ISMT.  For  analytical  purposes,  the 
shot  group  diameter  for  the  paper  targets  in  the  live  fire  event  was  calculated  by 
measuring  the  distance  between  the  outside  diameters  of  the  farthest  two  bullet 
intersections  on  the  target  and  subtracting  the  size  of  the  bullet  to  compensate  for  center 
of  bullet  impact  with  the  target.  For  the  ISMT  targets,  the  measurements  were  calculated 
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by  the  computer  software  where  the  system  recorded  the  laser  intersection  with  the  target 
and  displayed  the  distance  on  the  results  screen.  Therefore,  exploratory  analyses  were 
conducted  on  these  variables  at  the  three  different  distances  to  gain  greater  insight  into 
the  main  results.  All  exploratory  analyses  results  should  be  interpreted  with  caution  due 
to  the  small  sample  size  and  increased  probability  of  Type  I  error. 

1.  Control  Group  to  Simulation  Group  Comparisons  at  Each  Distance 

From  Target 

a.  MPI 

The  P-values  indicate  that  the  simulation  group  performed  better  than  the 
control  group  at  the  7-  and  15-yard  distances  during  post-training  and  at  the  3-yard 
distance  for  live  fire.  There  was  a  trend  for  the  simulation  group  to  show  less 
degradation  from  baseline  to  live  fire  (t(18)  =  1.46  p  =  0.08)  and  post-training  to  live  fire 
(7(18)  =  1.62,  p  =  0.06)  for  the  3-yard  distance  than  the  control  group.  The  simulation 
group’s  MPI  score  degraded  from  post-training  to  live  fire,  on  average,  by 
(0.29  ±  1.38  cm),  whereas  the  control  group  degraded  by  (2.30  ±  4.68  cm).  Similarly, 
from  baseline  to  live  fire,  the  simulation  group  degraded,  on  average,  by  only 
(0.09  ±  1.39  cm),  whereas  the  control  group  degraded  by  (1.99  ±  4.97  cm). 

Table  11  provides  means  and  standard  deviations  for  MPI  and  associated 
one  tail  P-value  at  each  distance  from  target  for  baseline,  post-training,  and  live  fire  by 
group. 
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Table  11.  Comparison  of  Control  and  Simulation  Groups  for  MPI  at  3,  7,  and  15  Yards 


Baseline 

I 

5ost-Training 

Live  Fire 

Control 

(n=16) 

Simulation 

(n-15) 

Control 

(n=16) 

Simulation 

(n=15) 

Control 

(n=16) 

Simulation 

(n=12) 

MPI  at 

3  yards 

Mean 

2.24 

2.35 

Mean 

1.93 

1.87 

Mean 

4.23 

2.01 

sd 

1.08 

1.45 

sd 

0.74 

0.92 

Sd 

4.48 

0.89 

P(T<=t; 

l  one -tail 

0.4065 

P(T<=t; 

l  one-tail 

0.4153 

P(T<=t) 

i  one-tail 

0.0349 

MPI  at 

7  yards 

Mean 

5.96 

5.72 

Mean 

4.47 

3.02 

Mean 

5.73 

4.45 

sd 

2.83 

2.43 

sd 

2.07 

1.45 

Sd 

4.28 

2.49 

P(T<=t)  one -tail 

0.4028 

P(T<=t)  one-tail 

0.0163 

P(T<=t) 

i  one-tail 

0.1642 

MPI  at 
15  yards 

Mean 

8.58 

8.50 

Mean 

6.71 

4.59 

Mean 

11.01 

8.90 

sd 

4.12 

3.29 

sd 

3.22 

2.17 

Sd 

5.28 

3.50 

P(T<=t)  one -tail 

0.4761 

P(T<=t)  one-tail 

0.0210 

P(T<=t) 

i  one-tail 

0.1203 

b.  Shot  Group  Diameter 

The  P-values  indicate  no  significant  differences  at  any  distance  from  the 
target  between  the  simulation  and  control  groups  for  shot  group  diameter. 

Table  12  provides  means  and  standard  deviations  for  shot  group  diameter 
and  associated  one  tail  P-value  at  each  distance  from  the  target  for  baseline,  post-training, 
and  live  fire  by  group. 


Table  12.  Comparison  of  Control  and  Simulation  Groups  for  Shot  Group  Diameters 

at  3,  7,  and  15  Yards 


Baseline 

Post-Training 

Live  Fire 

Control 

Simulation 

Control 

Simulation 

Control 

Simulation 

(n=16) 

(n-15) 

(n=16) 

(n-15) 

(n=16) 

(n-12) 

Shot 

Mean 

9.76 

8.47 

Mean  7.03 

6.31 

Mean  15.85 

13.64 

Group  at 

sd 

4.70 

4.05 

sd  3.29 

2.38 

Sd  7.13 

5.93 

3  yards 

P(T<=t)  one-tail 

0.2119 

P(T<=t)  one-tail 

0.2472 

P(T<=t)  one-tail 

0.1961 

Shot 

Mean 

20.31 

15.96 

Mean  15.24 

12.61 

Mean  25.05 

20.66 

Group  at 

sd 

14.12 

7.90 

sd  6.67 

6.52 

Sd  10.38 

7.24 

7  yards 

P(T<=t)  one-tail 

0.1505 

P(T<=t)  one-tail 

0.1388 

P(T<=t)  one-tail 

0.1109 

Shot 

Mean 

37.34 

32.31 

Mean  27.56 

26.98 

Mean  41.13 

40.45 

Group  at 

sd 

26.01 

14.78 

sd  17.14 

7.41 

Sd  12.19 

12.54 

15  yards 

P(T<=t)  one -tail 

0.2555 

P(T<=t)  one-tail 

0.4517 

P(T<=t)  one-tail 

0.4439 
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2.  Two-  and  Four-Week  Gaps  Between  Post-Training  and  Live  Fire 

Comparisons  at  Each  Distance  From  Target 

a.  MPI 

The  P-values  indicate  significant  differences  in  MPI  for  the  3-yard 
distance  from  the  target  in  the  baseline,  with  the  four-week  gap  group  having  a  smaller 
MPI  than  the  two-week  gap  group.  A  significant  difference  was  found  in  the  change  in 
MPI  between  post-training  and  live  fire  at  the  3-yard  distance.  The  two-week  gap  group 
showed  improvement  from  baseline  to  live  fire  (-0.11  ±  1.40  cm),  whereas  the  four- week 
gap  group  showed  degradation  from  baseline  to  live  fire  (2.66  ±  5.29  cm), 
(t(13)  =  -1.84,  p  =  0.045).  There  was  also  a  trend  towards  significance  where  the 
two-week  gap  group  showed  less  degradation  (0.50  ±  1.59  cm)  than  the  four-week  gap 
group  from  post-training  to  live  fire  (2.52  ±5.11  cm),  (r(14)  =  -1.36,  p  =  0.097). 

Table  13  provides  means  and  standard  deviations  for  MPI  and  associated 
one  tail  P-value  at  each  distance  from  the  target  for  baseline,  post-training,  and  live  fire 
by  time  gap. 


Table  13.  Comparison  of  Two-  and  Four-Week  Gaps  Between  Training  and  Live  Fire  MPI 

at  3,  7,  and  15  Yards 


Baseline 

Post- Training 

Live  Fire 

2  Weeks 
(n-16) 

4  Weeks 
(n—15) 

2  Weeks 
(n=16) 

4  Weeks 
(n= 15) 

2  Weeks 
(n=15) 

4  Weeks 
(n—13) 

MPI  at 

3  yards 

Mean 

2.69 

1.87 

Mean 

1.92 

1.88 

Mean 

2.43 

4.26 

sd 

1.17 

1.24 

sd 

0.75 

0.91 

Sd 

1.28 

4.99 

P(T<=t)  one-tail 

0.0336 

P(T<=t)  one-tail 

0.4489 

P(T<=t)  one -tail 

0.1110 

MPI  at 

7  yards 

Mean 

6.18 

5.47 

Mean 

3.33 

4.23 

Mean 

5.25 

5.10 

sd 

2.77 

2.45 

sd 

1.36 

2.33 

Sd 

3.82 

3.53 

P(T<=t)  one-tail 

0.2333 

P(T<=t)  one-tail 

0.1024 

P(T<=t)  one -tail 

0.4561 

MPI  at 
15  yards 

Mean 

9.23 

7.81 

Mean 

5.13 

6.27 

Mean 

10.70 

9.42 

sd 

3.32 

4.02 

sd 

2.59 

3.22 

Sd 

4.74 

4.64 

P(T<=t)  one-tail 

0.1446 

P(T<=t)  one-tail 

0.1421 

P(T<=t)  one -tail 

0.2396 

b.  Shot  Group  Diameter 

The  P-values  indicate  no  significant  differences  at  any  distance  from  the 
target  between  the  two-week  gap  group  and  the  four-week  gap  group  for  shot  group 
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diameter.  There  were  trends  towards  significance  between  the  post-training  and  live  fire 
at  the  15 -yard  distance  from  the  target,  where  the  four- week  gap  group  had  less 
degradation  in  shot  group  diameter  (10.12  ±  17.12  cm)  than  the  two-week  gap  group 
(19.59  ±  12.11  cm),  (t( 24)  =  1.65,  p  =  0.056);  the  baseline  and  live  fire  at  the  3-yard 
distance  from  the  target,  where  the  four-week  gap  group  had  less  degradation  in  shot 
group  diameter  (4.35  ±  2.79  cm)  than  the  two-week  gap  group  (6.96  ±  6.90  cm), 
(t(  19)  =  1.34,  p  =  0.097);  and  the  baseline  and  live  fire  at  the  15-yard  distance  from  the 
target,  where  the  four- week  gap  group  had  less  degradation  in  shot  group  diameter 
(0.80  ±  26.05  cm)  than  the  two-week  gap  group  (11.96  ±  16.28  cm),  (t( 24)  =  1.33, 
p  =  0.098). 

Table  14  provides  means  and  standard  deviations  for  shot  group  diameter 
and  associated  one  tail  P-value  at  each  distance  from  the  target  for  baseline,  post-training, 
and  live  fire  by  time  gap. 


Table  14.  Comparison  of  Two-  and  Four-Week  Gaps  Between  Training  and  Live  Fire 

of  Shot  Group  Diameters  at  3,  7,  and  15  Yards 


Baseline 

Post-Training 

Live  Fire 

2  Weeks 

4  Weeks 

2  Weeks 

4  Weeks 

2  Weeks 

4  Weeks 

(n=16) 

(n=15) 

(n=16) 

(n=15) 

(n=15) 

(n=13) 

Shot 

Mean 

9.15 

9.12 

Mean  7.19 

6.13 

Mean 

16.32 

13.26 

Group 

sd 

3.33 

5.40 

sd  3.53 

1.89 

Sd 

7.51 

5.23 

at  3 
yards 

P(T<=t)  one -tail 

0.4927 

P(T<=t)  one -tail 

0.1506 

P(T<=t)  one-tail 

0.1148 

Shot 

Mean 

18.48 

18.05 

Mean  12.56 

15.47 

Mean 

22.25 

24.24 

Group 

sd 

11.22 

12.57 

sd  4.28 

8.35 

Sd 

8.14 

10.68 

at  7 
yards 

P(T<=t)  one -tail 

0.4614 

P(T<=t)  one -tail 

0.1205 

P(T<=t)  one-tail 

0.2902 

Shot 

Mean 

32.64 

37.33 

Mean  24.73 

29.99 

Mean 

42.65 

38.75 

Group 

sd 

13.79 

27.23 

sd  6.62 

17.54 

Sd 

12.46 

11.83 

at  15 
yards 

P(T<=t)  one -tail 

0.2777 

P(T<=t)  one -tail 

0.1447 

P(T<=t)  one-tail 

0.2026 
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IV.  SUMMARY,  DISCUSSION,  RECOMMENDATIONS, 

AND  CONCLUSIONS 

A.  SUMMARY 

This  thesis  examined  how  simulation-based  marksmanship  training,  using  ISMT, 
transfers  to  live  fire  compared  to  SNMT,  and  how  long  any  beneficial  training  effects 
last.  This  research  explored  whether  the  use  of  simulation,  combined  with  live  fire,  can 
provide  a  more  robust  method  for  conducting  marksmanship  training  than  SNMT.  The 
research  questions  addressed  in  this  thesis  were:  (1)  Does  ISMT  accomplish  skill  transfer 
from  the  virtual  environment  to  actual  proficiency  in  marksmanship  performance?  (2) 
Can  the  ISMT  be  used  as  an  effective  part-task  trainer  to  improve  performance  with  the 
M9  on  the  NHQC?  Specifically,  will  participants  in  the  simulation  group  have  a  better 
chance  of  retaining  what  they  learned  after  two  weeks  or  four  weeks  of  no  instruction, 
than  participants  in  the  control  group  when  doing  live  fire? 

The  experimental  design  for  this  thesis  utilized  a  between-groups  study  of  active 
duty  military  volunteers  randomly  selected  to  complete  either  control  group  or  simulation 
group.  Marksmanship  performance  was  measured  by  MPI  of  group  shots  and  scores  on 
the  NHQC.  Prior  to  the  first  session,  all  participants  completed  a  demographic  survey 
containing  questions  that  asked  about  the  participants’  characteristics,  marksmanship  skill 
level,  and  experience.  The  first  session  consisted  of  recording  a  baseline  of  their  current 
level  of  marksmanship  performance  and  score  recorded  via  the  ISMT  before  exposure  to 
either  training  condition.  Control  groups  received  SNMT,  and  simulation  groups 
received  training  using  the  feedback  tools  from  ISMT.  Both  groups  participated  in  an 
additional  post-training  simulation  NHQC  firing  sequence.  One  control  and  simulation 
group  did  not  receive  any  additional  training  for  two  weeks,  while  the  other  control  and 
simulation  group  did  not  receive  any  additional  training  for  four  weeks.  After  their 
respective  time  lapse,  all  groups  conducted  a  final  live  fire  NHQC  firing  sequence  to 
determine  their  retention  of  marksmanship  skill. 
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B.  HYPOTHESIS  ONE  DISCUSSION 


HOi:  There  will  be  no  group  differences  in  the  participants’  MPI  of  shots  and 
scores  in  the  NHQC  firing  sequence  from  baseline  to  live  fire. 

HAp  Participants  who  receive  simulation-based  training  in  ISMT  will  have  a 
greater  improvement  in  MPI  of  shots  and  scores  in  the  NHQC  firing  sequence  when 
comparing  the  difference  between  baseline  and  live  fire  than  participants  who 
receive  SNMT. 

The  analysis  indicated  that  there  were  no  significant  differences  in  performance  in 
scores  or  MPI  between  the  simulation  group  and  control  group  in  any  of  the  shooting 
events.  Important  details  from  this  analysis  will  be  discussed  below. 

1.  Scores 

Both  the  simulation  and  control  groups  significantly  improved  scores  from 
baseline  to  live  fire;  however,  there  were  no  statistical  differences  rate  of  improvement 
between  the  two  groups.  Based  on  previous  research  results,  which  showed  simulation 
training  performed  no  worse  than  live  training,  the  results  of  this  portion  of  the  analysis 
were  not  surprising.  Score  performance  is  dependent  only  on  where  each  shot  hits  the 
Transtar  II  target  and  is  not  based  on  the  grouping  of  shots.  Examining  the  Transtar  II 
target,  region  one  yields  zero  points,  region  two  yields  two  points,  region  three  yields 
three  points,  region  four  yields  four  points,  and  regions  five  and  six  yield  five  points 
(see  Figure  11).  Although  the  comparison  of  difference  scores  between  groups  did  not 
show  statistical  differences,  it  is  important  to  note  that  the  control  group  average  baseline 
score  was  201.44  ±  21.25  points,  whereas  the  simulation  group  average  baseline  score 
was  210.33  ±  21.94  points.  This  difference  is  important  because  there  is  not  as  much 
room  for  improvement  for  the  simulation  group’s  score  due  to  the  maximum  allowable 
score  of  240  points.  Indeed,  4  of  the  15  simulation  participants  were  close  to  the 
maximum  score  at  baseline,  compared  to  2  of  the  16  control  participants.  Thus,  the  trend 
for  the  control  group  to  show  greater  improvement  in  their  scores  could  be  due  to  the 
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control  group  having  more  room  to  improve.  A  future  study  that  more  carefully  matches 
the  baseline  marksmanship  performance  between  simulation  and  control  participants  is 
necessary  to  investigate  this  suggestion. 


Region  1  Outside  of  Target  Silhouette 


Figure  11.  Transtar  II  Silhouette  Target  with  Regions  (From  Getty,  2008) 


2.  MPI 

The  analysis  indicated  significant  between  group  differences  in  change  in  the  MPI 
performance  from  baseline  to  live  fire  in  which  the  simulation  group  showed  no 
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significant  change  in  performance,  but  the  control  group  showed  degradation  in 
MPI  measurements. 

Because  regions  five  and  six  on  the  Transtar  II  target,  approximately  (30.8  cm  tall 
and  20.6  cm  wide)  ellipse,  are  both  scored  as  five  points  per  shot,  the  score  and  MPI  do 
not  directly  parallel  each  other  in  results.  In  other  words,  the  MPI  is  a  more  fine-tuned 
measure  of  performance.  The  results  suggest  that  the  simulation  group  not  only  shot  as 
well  as  the  control  group,  but  their  MPI  was  less  likely  to  deteriorate  over  time  compared 
to  the  control  group. 

C.  HYPOTHESIS  TWO  DISCUSSION 

HOi:  Participants  in  each  group  will  be  equally  likely  to  maintain  their  MPI  of 
shots  and  scores  on  the  NHQC  two  weeks  and  four  weeks  after  the  training  day. 

HAi:  Participants  who  receive  simulation-based  training  in  ISMT  will  be  more 
likely  than  those  in  the  control  group  to  maintain  their  MPI  of  shots  and  scores  in  the 
NHQC  two  weeks  and  four  weeks  after  the  training  day. 

Hypothesis  two  was  not  able  to  be  tested  as  written  due  to  the  small  sample  sizes 
for  each  condition.  Instead,  data  from  the  training  groups  were  combined  and  the  main 
effects  of  time  gap  were  assessed.  Thus,  the  two-week  group  had  seven  simulation  group 
participants  and  eight  control  group  participants,  and  the  four-week  group  had  five 
simulation  group  participants  and  seven  control  group  participants. 

The  analysis  indicates  that  there  was  no  significant  difference  in  change  in 
performance  in  scores  or  MPI  from  baseline  to  live  fire  between  participants  with  a 
two-week  gap  from  baseline  to  live  fire  than  participants  with  a  four-week  gap. 
Important  details  from  this  analysis  will  be  discussed  below. 

1.  Scores  and  MPI 

The  analysis  indicated  that  there  were  no  significant  differences  found  in  the 
difference  score  and  MPI  between  the  two-  and  four-week  gap  groups  from  baseline  to 
live  fire.  The  results  are  not  surprising  given  that  Getty  (2008)  concluded  that  a  one- 
week  gap  was  not  enough  time  to  determine  retention.  The  initial  experimental  design 
for  this  research  was  for  shooters  to  have  a  live  fire  event  the  week  following  training, 
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followed  by  either  a  two-  or  four-month  gap  between  the  initial  live  fire  event  and  a  final 
live  fire  event.  However,  due  to  time  constraints,  the  initial  live  fire  event  following 
training  had  to  be  removed  and  the  time  gaps  between  post-training  and  live  were  limited 
to  two  and  four  weeks. 

The  analysis  of  the  two-  and  four-week  time  gaps  was  insufficient  to  show  any 
skill  degradation.  We  were  unable  to  test  whether  there  were  training  group  x  time  gap 
interactions  due  to  the  small  sample  sizes.  A  future  study  that  has  adequate  sample  sizes 
for  each  of  these  conditions  is  needed  to  address  this  research  question.  Additionally, 
longer  time  gaps  are  required  to  detect  degradation.  Time  gaps  of  3  and  6  months  or  6 
and  12  months  should  be  investigated. 

D.  EXPLORATORY  ANALYSIS  DISCUSSION 

As  noted  in  Chapter  III,  each  shooter  participated  in  an  initial  baseline  shooting 
event,  a  post-training  shooting  event,  and  a  final  live  fire  shooting  event.  Exploratory 
analysis  was  conducted  on  the  shot  group  diameter  and  MPI  measurements  and  was 
recorded  for  each  firing  distance  from  the  target  (3,  7,  and  15  yards). 

1.  Control  Group  to  Simulation  Group  Comparisons  at  Each  Distance 

Lrom  Target 

a.  MPI 

The  analysis  indicated  that  the  simulation  group  performed  better  than  the 
control  group  at  the  7-  and  15-yard  distances  during  post-training  and  at  the  3-yard 
distance  for  live  fire.  These  results  suggest  that  ISMT  training  may  be  most  beneficial 
for  the  7-  and  15 -yard  distances  and  simulation  to  live  fire  training  transfer  occurred  for 
the  3-yard  distance.  These  results  pinpoint  the  distances  in  which  simulation  training 
appears  to  be  more  beneficial  than  SNMT. 

b.  Shot  Group  Diameter 

The  analysis  indicated  no  significant  differences  at  any  distance  from  the 
target  between  the  simulation  and  control  groups  for  shot-group  diameter. 
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2.  Two-  and  Four-Week  Gaps  Between  Post-Training  and  Live  Fire 

Comparisons  at  Each  Distance  From  Target 

a.  MPI 

The  analysis  indicated  significant  differences  in  MPI  for  the  three-yard 
distance  in  the  baseline,  with  the  four-week  gap  group  having  a  smaller  MPI  than  the 
two-week  gap  group.  This  result  was  surprising  given  the  expectation  of  performance 
degradation  over  time.  Also,  for  the  change  in  MPI  from  baseline  to  live  fire  comparison 
at  the  3-yard  distance,  the  two-week  gap  group  maintained  their  MPI  better  than  the 
four-week  gap  group.  As  stated  above,  larger  sample  sizes  are  needed  to  assess  a 
training  group  by  time  gap  interaction  effect.  For  example,  this  result  may  be  due  to  the 
simulation  group  showing  steady  MPI  performance  at  both  two  weeks  and  four  weeks, 
whereas  the  control  group  showed  degradation  at  four  weeks. 

b.  Shot  Group  Diameter 

The  analysis  indicated  no  significant  differences  at  any  distance  from  the 
target  between  the  simulation  and  control  groups  for  shot-group  diameter. 

E.  FUTURE  WORK  AND  RECOMMENDATIONS 

This  research  originally  contained  two  live  fire  events,  but  due  to  time  constraints, 
the  research  design  was  modified  to  incorporate  only  one  live  fire  event.  A  future  study 
should  follow  the  original  design  of  this  research  to  include  a  pre-training  demographic 
questionnaire  and  a  baseline  shooting  event  in  ISMT  of  all  participants,  followed  by 
experimental  training  in  ISMT  with  a  post-training  shooting  event  in  ISMT  and  control 
training  using  SNMT  with  a  post-training  shooting  event  in  ISMT.  The  week  following 
completion  of  training,  both  experimental  and  control  groups  should  complete  a  live  fire 
shooting  event  using  the  NHQC  firing  sequence.  After  the  initial  live  fire  event,  a  longer 
time  gap  (i.e.,  2-  and  4-month,  3-  and  6-month,  or  6-  and  12-month  gap)  than  what  was 
used  in  this  research  could  be  beneficial.  In  addition,  using  a  larger  sample  size  of 
participants  could  yield  more  definitive  data,  particularly  regarding  the  time  gap  results. 

A  future  study  also  should  explore  whether  simulation  marksmanship  training  is 
more  likely  to  reduce  the  effects  of  performance  anxiety  in  a  live  fire  shooting  event  than 
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SNMT.  Chung,  Delacruz,  Vries,  Bewley,  and  O’Neil  (2005)  found  that  anxiety  had  a 
significant  negative  effect  on  marksmanship  performance.  Their  goal  was  to  measure 
how  anxiety  affected  novice  shooters’  performance  and  to  what  degree  anxiety 
contributed  to  the  prediction  of  performance  above  and  beyond  aptitude.  They  used  the 
annual  qualification  of  Marines  in  rifle  marksmanship  as  the  event,  since  the  qualification 
is  extremely  stressful  for  the  participating  Marines  because  individual  scores  directly 
affect  promotion.  The  annual  qualification  consists  of  a  week  of  instruction  and  practice, 
culminating  on  the  last  day  with  the  final  qualification.  The  importance  of  this  study  was 
clear  when  the  final  qualification  indicated  that  overall  state  anxiety  and  state  worry  were 
negatively  correlated  with  firing  line  experience.  Participants  who  had  higher  levels  of 
anxiety  also  had  higher  levels  of  worry.  Participants  who  had  a  more  positive  firing  line 
experience  had  lower  anxiety  and  worry.  A  research  design  similar  to  this  study  should 
be  used,  while  also  incorporating  both  self-report  and  physiological  measures  of  anxiety, 
such  as  heart  rate  and  galvanic  stress  response  during  the  ISMT  shooting  event  and  live 
fire  shooting  event.  A  supplement  to  questionnaires  would  include  monitoring  heart  rate 
during  the  ISMT  shooting  event  and  live  fire  shooting  event. 

During  the  course  of  this  research,  only  one  weapon  malfunction  occurred 
compared  to  nine  weapons  malfunctions  experienced  by  Getty  (2008).  The  lower 
number  of  malfunctions  may  be  attributed  to  shutting  down  the  ISMT  system  between 
participants  and  allowing  the  laser  inside  each  weapon  to  cool.  By  minimizing  weapons 
malfunctions,  this  eliminated  prolonged  down  time  of  ISMT  while  a  new  weapon  was 
shipped  to  NPS. 

F.  CONCLUSION 

The  results  of  this  research  indicates  that:  (1)  simulation  training  leads  to 
marksmanship  performance  no  worse  than  SNMT  regarding  scores  and  (2)  simulation 
training  leads  to  marksmanship  performance  that  is  better  than  SNMT  regarding  MPI. 
MPI  provides  a  more  accurate  measure  of  marksmanship  performance  than  score  and, 
therefore,  may  be  more  indicative  of  how  well  a  watchstander  would  perform  in  an  actual 
threat  situation.  ISMT  provides  a  medium  of  training  that  is  more  versatile  than  live 
training  in  that  it  can  be  accomplished  in  any  weather  conditions  because  it  is  an  indoor 
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trainer.  The  Navy’s  use  of  ISMT  onboard  ships  would  provide  the  means  for  Sailors  and 
Marines  to  practice  a  perishable  skill,  such  as  marksmanship,  regardless  of  weather,  ship 
schedule,  or  operational  commitments  while  underway.  These  factors  often  inhibit  live 
fire  training.  Having  an  ISMT  onboard  ship  also  would  allow  Sailors  and  Marines  to 
meet  the  current  sustainment  training  requirement  of  eight  months  and  could  allow  extra 
training,  with  a  substantially  reduced  cost  compared  to  live  fire  training.  Personnel  could 
train  and  practice  often,  honing  marksmanship  skills,  while  ultimately  providing 
better-trained  watchstanders  and  increasing  the  marksmanship  skills  of  ships 
security  force. 


52 


LIST  OF  REFERENCES 


Chief  of  Naval  Operations.  (2009,  August  12).  Small  arms  training  and  qualification 
(OPNAV  INSTRUCTION  3591. IF).  Washington,  D.C.:  Walsh,  P.  M. 

Chung,  G.  K„  Delacruz,  G.  C„  Vries,  L.  F„  Bewley,  W.  L.,  &  O’Neil,  H.F.  (2005).  The 
role  of  anxiety  on  novice’s  rifle  marksmanship  performance.  Educational 
Assessment ,  10(3),  257-275. 

Faram,  M.  D.,  Fellman,  S.,  Fuentes,  G.,  McMichael,  W.  H.,  Maze,  R.,  Stewart,  J.  (2012, 
January  9).  17  Things  you  should  know  about  2012.  Navy  Times ,  61(16),  18-21. 
Retrieved  from  http://navytimes.va.newsmemory.com/default.php?token= 
a5cb686fb41ade6aabe769albe87577c&pSetup=navytimes 

Getty,  T.  J.  (2008).  A  comparison  of  current  naval  marksmanship  training  vs. 

simulation  based  marksmanship  training  with  the  use  of  indoor  simulation 
marksmanship  trainer  (ISMT)  (Master’s  thesis).  Monterey.  CA:  Naval 
Postgraduate  School.  Manuscript  in  preparation. 

Headquarters,  United  States  Marine  Corps.  (2003,  November).  Pistol  marksmanship 
(MCRP3-01B).  Washington,  D.C.:  Hanlon,  E.  J. 

Joint  Chiefs  of  Staff.  (2012,  January).  Department  of  Defense  Dictionary  of  Military 
and  Associated  Terms  (JP  1-02).  Washington,  DC:  W.  E.  Gortney.  Retrieved 
from  http://www.dtic.mil/doctrine/new_pubs/jpl_02.pdf 

Naval  Surface  Forces  Command.  (2008,  September).  Antiterrorism/force  protection 

program  (COMNAVSURFORINST  3300. IB).  San  Diego,  CA:  Gumataotao,  P. 
A. 

U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences.  (2000,  July). 

Basic  rifle  marksmanship  training  with  the  laser  marksmanship  training  system 
(Research  Report  1761).  Alexandria,  VA:  Hagman,  J.  D.  Retrieved  from 
http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA380109 

U.S.  Army  Research  Laboratory.  (2007,  September).  A  comparison  of  live  and 

simulated  fire  soldier  shooting  performance  (ARL-TR-4234).  Aberdeen  Proving 
Ground,  MD:  Scribner,  D.  R.,  Wiley,  P.  H.,  &  Harper,  W.  H.  Retrieved  from 
http  ://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA47 1786 

U.S.  Fleet  Forces  Command.  (2003,  August).  Force  protection  weapons  handling 

standard  procedures  and  guidelines  (NTRP  3-07.2.2).  Norfolk,  VA:  Route,  R.  A. 


53 


U.S.  Marine  Corps  Concepts  &  Programs.  (2008).  Indoor  simulated  marksmanship 

trainer-enhanced  (ISMT-E).  Retrieved  from  http://www.marines.mil/unit/pandr/ 

Documents/Concepts/2008/PDF/CP08Ch3P7_Indoor_Simulated_Marksmanship_ 

Trainer%E2%80%93Enhanced.pdf 

Yates,  W.  W.  (2004).  A  training  transfer  study  of  the  indoor  simulated  marksmanship 
trainer  (Master’s  thesis).  Monterey,  CA:  Naval  Postgraduate  School.  Retrieved 
from  http://edocs.nps.edu/npspubs/scholarly/theses/2004/Sep/04Sep_Yates.pdf 


54 


INITIAL  DISTRIBUTION  LIST 


1.  Defense  Technical  Information  Center 
Ft.  Belvoir,  Virginia 

2.  Dudley  Knox  Library 
Naval  Postgraduate  School 
Monterey,  California 


55 


